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Synopsis 
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Discrete Cosine Transform (DCT) based image compression with its well known 
advantages is being used in various applications. Especially, most of the international 
standards for image and video compression such as JPEG, H.261, H.263 and MPEG 
have recommended the use of block DCT as a main compression technique. The DCT 
scheme has potential to be used as a compression scheme because of its desirable energy 
compactness property and relative ease of implementation. The performance of such a 
system at high compression ratio is limited by blocking (irtijacts. 

In DCT based compression schemes, the high, corapiession ratios are obtained by 
discarding information about DCT coefficients that are considered to be less important. 
This results in visible discontinuities along the block boundaries, commonly known as 
blocking artifacts. These artifacts often limit the maximum compression ratios that 
could be achieved. One main reason of blocking artifacts is that the blocks are en- 
coded and quantized independently without considering the correlation between adjacent 
blocks. These blocking artifacts cause three types of noise: grid noise in monotone area, 
staircase noise near the edges and corner outliers at the cross point of M x M blocks. 
Noise in block coding is correlated with the local chara-cteristics of signal. Grid noise 
in monotone area, causing visually annoying artifacts, manifests itself as an artificial 
boundary among the pixels of adjacent blocks. An edge usually partitions an image into 
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two monotone regions. Block coder can maintain continuity of edges within the blocks 
but cannot assure this across them. Furthermore, if the coder is not able to adequately 
represent the part of an edge within a block, then the degradation of an entire edge 
is increased. Therefore, edges tend to unsmooth in block boundaries and each side of 
an edge is no longer a monotone region. This is known as staircase noise. The ringing 
further increases the problems in edge area. Ringing is well known Gibbs phenomenon 
due to the truncation of high frequency components by quantization. 

Postprocessing of the DCT coded the images can reduce the coding artifacts and 
therefore it is a very promising research area. The main objective of postprocessing 
is to remove the coding artifacts with lowest computational complexity. As discussed 
above, the noise in block-coded images is correlated with the local characteristics of the 
signal; so space-invariant filters are unable to exploit the correlation to reduce the noise. 
These filters are not suitable because they will blur the edge and degrade the visual 
quality of image. This is because most of the grid noise is out-of-band, while staircase 
noise is in the direction parallel to the edge. Signal bandwidth in monotone area is 
much lower than the bandwidth of the average spectrum of the edge blocks. Thus, this 
filter cannot remove the locally out-of-band noise in monotone area without blurring the 
edges. It is also found that the Human Visual System (HVS) is more sensitive to blocking 
artifacts in monotone regions than the edge regions. Thus, effect of noise in edge area 
reduces due to masking efiect. A strong smoothing filter is required in monotone area 
but this smoothing operation in edge region tends to introduce undesirable blur. For 
edge regions, smoothing of a few pixels around block boundaries by directional lowpass 
filter is sufficient. Hence, space-variant or adaptive filtering depending on local image 
characteristics is preferable. The information about local image characteristics can be 
obtained from the image by edge detection or measuring the flatness of the region. Thus, 
one of the best approach for postprocessing is to classify the image blocks into various 
categories and then each is processed by one or two-dimensional (2D) fiOiter. 

Several postprocessing algorithms based on space- variant/ adaptive filtering have 
been proposed to reduce the blocking effect of block coding system [1-4]. As discussed, 
linear space-invariant filtering is inadequate to remove these artifacts. Iterative methods 
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such as Projection Onto Convex Set (POCS) and Constrained Least Square (CLS) are 
not suitable for real-time applications because of their greater computational complexity. 
Space-variant/ adaptive filtering schemes require the edge information extracted from 
the received blocky images which contain false edges due to the blocking artifacts. 
Recovery of loss of accuracy of DCT coefficient by discontinuity criterion is another 
method [5] to reduce the artifacts. But this method provides good results only in the 
monotone area. Thus, the algorithms proposed in the literature are not able to reduce 
the coding artifacts in various regions of image simultaneously with low computational 
complexity. Therefore, they are not suitable for real time applications. The performance 
of DCT based image compression scheme degrades at high compression ratios and most of 
the existing postprocessing algorithms are not able to give good results in this situation. 
Therefore,, postprocessing algorithms with good performance at high compression ratios 
are much needed. 

^This thesis addresses various postprocessing issues that may arise in practice: (i) 
reduction of coding artifacts; and (fi) computationally efficient methods to make it 
suitable for real time applications such as video and still image decoding. The coding 
artifacts are more visible at higher compression ratios but they can be reduced more 
efficiently than the existing postprocessing techniques. Various aspects of DCT based 
image compression and postprocessing methods are studied with a state-of-art survey 
on the subject. A number of postprocessing algorithms for reducing the coding artifacts 
are being proposed in this work./- 

li'he first contribution of the thesis is the development of a hybrid scheme for post- 
processing by clubbing the merits of several existing algorithms. Smoothing of artificial 
discontinuities due to blocking artifacts improves the image quality whereas smoothing 
of actual image edges degrades it^ Thus, there is a need to satisfy these two conflicting 
requirements as simply as possible. The proposed algorithm includes the advantage 
of adaptive filtering in ed^ area and block-boundary discontinuity minimization in 
monotone area along with the comer outliers reduction. 

The second contribution of the thesis is the design of a 2D multiple-notch (Both FIR 
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and HR) filter and its applications in reducing the blocking artifacts. These filters are 
designed with the goal of reducing blocking artifacts from DCT coded images. 

The third contribution of the thesis is to provide four computationally efficient 
algorithms for postprocessing of DCT coded images. In literature several powerful 
postprocessing algorithms have been developed to reduce blocking artifacts. However, 
from application point of view, these are complex for real time applications such as 
video and still image decoding. These proposed algorithms can improve visual quality 
with improvement in peak signal-to-noise ratio (PSNR) with a very low computational 
complexity. The performance analysis and the simulation results show that the proposed 
algorithms are very competitive compared with those available in the literature.^ 

Summary of algorithms proposed in this work 

In the present work, a number of algorithms for postprocessing of DCT coded images 
have been proposed. These algorithms can be summarized as: 

A hybrid scheme for the postprocessing of DCT coded images is proposed. The 
scheme is named as hybrid as the processing is done in both DCT as well as in spatial 
domain by two different algorithms. The scheme makes use of adaptive filtering in edge 
area and block-boundary discontinuity minimization in monotone area. The use of a 
lower order signal adaptive filter and the block-boundary discontinuity minimization al- 
gorithm reduces the processing time. Corner outliers reduction scheme further improves 
the performance. In this scheme, the compensating DCT coefficients for monotone block 
are calculated and added to corresponding DCT coefficients to achieve minimum block- 
boundary discontinuity. These compensating DCT coeffi.cients for monotone blocks are 
calculated from the DCT of the block-boundary pixel differences. In this calculation, 
some of the block-boundary pixel differences are made zero if the neighboring block is 
an edge block. In the proposed scheme, the edge information is extracted and used for 
the classification of image blocks. For edge blocks, a signal adaptive filter performs the 
filtering along the edge to reduce the staircase noise. This filter also uses the same edge 
information (edge-map). It has been observed that the performance of this algorithm is 
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better than the space variant filtering as well as the two individual algorithms used in 
the hybrid scheme. 

We have developed the design method for 2D HR multiple-notch filter and some of 
its applications are being proposed here. A simple algebraic method [6] is used for 
the design of this filter. In this design approach, first two ID multiple-tone filters axe 
designed as per the specifications of 2D multiple-notch filter and then these two filters 
are used to obtain 2D multiple-vertical line filter and 2D multiple-horizontal line filter. 
Finally, cascade of these two filters gives the desired 2D multiple-notch filter. The design 
of ID multiple- tone filter is based on the design of allpass filter as discussed in [7]. Then 
two new applications of 2D multiple-notch filter is being suggested. The possibility 
of using 2D multiple-notch filter for reducing blocking artifact is fully examined. It 
has been shown that 2D multiple-notch filter can eliminate the square grid. It can also 
reduce the blocking artifacts from DCT coded images. The concept of reducing blocking 
artifacts by 2D multiple-notch filter is based on the fact that the discontinuities due to 
block artifacts in block decoded image have periodic structure which ultimately results 
in some peaks in the DCT domain. These discontinuities are highlighted in the gradient 
of the image and thus dominant peaks can be observed in the DCT of this gradient- 
image. A 2D multiple-notch filter can kill these peaks to remove the false edges from 
the edge-map of blocky image. In addition to this, as the discontinuities due to blocking 
artifacts are more visible in the monotone area, the filter can be applied directly in these 
areas. Thus, improvement in the performances of space- variant/adaptive filter by the 
removal of false edges from the edge-map and the reduction of blocking artifacts directly 
in the monotone area, are the new possible applications of 2D multiple-notch filter. 

We have extended the design approach of 2D multiple-notch filter to FIR type. FIR 
filters axe suitable for image processing applications as they cause less distortion due to 
their linear phase property. The drawback of FIR filter is that they cause more delay. 
The design procedure for the 2D FIR multiple-tone filter is similar to IIR type of this 
filter but the desired 2D multiple-notch filter is obtained as a complementary filter [9] of 
the 2D multiple-tone filter. In this case, two ID FIR multiple-tone filters are designed 
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by windowed Fourier series method with Kaiser window. Various plots show that all 
ID and 2D filters are obtained as per the design approach. This filter is also applied 
for the reduction of blocking artifacts and removal of false edges from edge of blocky 
decompressed image. The results obtained in case of FIR filter are better than HR filter, 
as expected. 

At the end of this work, four simple and effective algorithms for the postprocessing 
of DOT coded images are being proposed. Although these algorithms look simple but 
they are able to reduce blocking artifacts significantly without much computational 
complexity. The performance of these computationally efficient algorithms is better at 
high compression ratios; therefore these are suitable for low bit rate video and image 
decoding applications. 

First algorithm proposed in this part of the work reduces the blocking artifacts by 
exploiting the residual inter-block correlation that exists in the received blocky image. 
For this purpose a signal adaptive filter is used to smoothen out a sub-image of DC 
coefficients of all the blocks. The algorithm reduces the blocking artifacts with a very 
low computational complexity. The second algorithm reduces the blocking artifacts 
significantly by capturing the discontinuities due to blocking artifacts in the new blocks 
(made by the re-division of the image) and removing them in the DCT domain by se- 
lective attenuation of AC components corresponding to these discontinuities. These two 
algorithms together reduce the blocking artifacts considerably with slight improvement 
in PSNR. It has been verified by calculating the various discontinuity measures. In the 
third algorithm, the discontinuities are captured in the same way but the singular value 
decomposition (SVD) based filtering scheme is used for their reduction. This algorithm is 
very effective for reducing blocking artifacts but its computational complexity is slightly 
more because SVD algorithm is applied to each monotone block. Fourth algorithm 
proposed here is again a DCT domain algorithm. It reduces the blocking artifacts by 
eliminating some selected DCT coefficients of concatenated block of two adjacent blocks. 
The algorithm has low computational complexity, because no filtering is required. The 
availability of fast algorithm for DCT computation makes this algorithm suitable for 
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low bit rate video and image decoding applications. In this algorithm, only the first 
row and first column of concatenated block are selected for elimination, which further 
reduces the computation time. The results show that this algorithm provides very good 
performance with minimum computational complexity. 

Motivations 

Therefore the motivations behind the work presented in this thesis are: 

• Uncompressed video and audio data are huge and there is a big problem for their 
storage and network communication. Therefore data compression is required. As 
the compression ratio of lossless methods (e.g., Huffman, Arithmetic) is not high 
enough for image and video compression, a lossy compression scheme that can 
provide higher compression ratio is needed. 

• Very, good energy compaction and decorrelation properties of DCT motivated to 
work on the DCT based image compression scheme. The performance of DCT 
based compression scheme is limited by blocking artifacts. So, the main objective 
was to find a DCT based scheme that can minimize these blocking artifacts. 

• Most of the existing popular standards for image compression also use DCT based 
image compression schemes. This was also one of the main reasons for taking up 
this problem. Although at the beginning a number of possibilities of modified 
coding schemes are studied but looking at these existing standards the work is 
centered around the postprocessing of DCT coded images. 

• DCT-based image compression is very popular and finds various applications. 
Improvement in quantization, encoding and postprocessing can increase the ca- 
pabilities of DCT-based image compression scheme over other schemes in many 
applications. 

• Low bit rate coding applications motivate to find computationally efficient algo- 
rithms for postprocessing of such images. 

Organization of Thesis 

Chapter 1 introduces various aspects of data compression with an overview of DCT 
based image compression scheme. Various methods of postprocessing of DCT coded 



XI 


images are discussed and a state-of-the-art survey on the subject is presented. 

In Chapter 2, a hybrid scheme is presented which takes advantages of adaptive 
filtering in edge area and block boundary discontinuity algorithm in monotone area. 

Chapter 3 presents an approach based on space- variant/ adaptive filtering. In this 
approach, a 2D HR multiple-notch filter is designed from ID multiple-notch filter. 
This filter is applied to reduce the blocking artifacts from DCT coded images either 
by improving the classification scheme or by directly filtering the blocking artifacts in 
monotone areas. 

In Chapter 4, the design approach presented in Chapter S of 2D multiple-notch filter 
is extended to FIR type filter. This filter is also applied to reduce the blocking artifacts 
from DCT coded images as in Chapter 3. 

Chapter 5 is devoted to some new computationally efficient algorithms. Four algo- 
rithms to reduce the artifacts in DCT domain are presented. 

Chapter 6 concludes the major contributions of this thesis and suggests the scope for 
future research work in this area. 
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Chapter 1 


Introduction 


1.1 Data Compression 

Digital images have become an important source of information in the modem world 
of communication systems. In their raw form, digital images require a huge amount 
of memory. Many research efforts have been made in solving the problem of image 
compression in the last two decades. Image compression is an extremely important 
part of modern computing. Due to the ability to compress images to a fraction of their 
original size, valuable and expensive disk space can be saved. In addition, transportation 
of images from one computer to another becomes easier and less time consuming. 
Therefore, image compression has played such an important role in the development 
of the Internet. 

Image, video, and audio signals can be compressed because there is considerable sta- 
tistical redundancy in the signal due to the existence of spatial and temporal correlation. 
There is also considerable information in the signal that is irrelevant from a perceptual 
point of view. Compression scheme may use these factors to achieve the desired data 
rate. 

The compression algorithms can be divided into two categories: lossless and lossy. 
Lossless compression is achieved if no distortion is introduced in the coded image. 
Predictive methods are suited to lossless and low-compression applications. A lossless 
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compression means that the restored data file is identical to the original. This is abso- 
lutely necessary for many types of data, for example: executable code, word processing 
files, tabulated numbers, etc. In this type of information we cannot afford to misplace 
even a single bit. In comparison, data files that represent images and other acquired 
signals do not need to be kept in perfect condition for storage or transmission. All 
real world measurements inherently contain a certain amount of noise. If the changes 
made to these signals resemble a small amount of additional noise, no harm is done. 
Compression techniques that allow this type of degradation are called lossy. The lossy 
techniques are much more effective for compression than lossless methods. 

At higher compression ratio the noise added to the data is more. Images transmitted 
over the World Wide Web are an excellent example of why data compression is important, 
and the effectiveness of lossless versus lossy compression. Suppose we need to download 
a detailed color image over a computer’s 33.6 kbps modem. If the image has not been 
compressed, it will contain about 504 Kbytes of data. If it has been compressed using a 
lossless technique, it will be about one-half this size, or 252 Kbytes. If lossy compression 
has been used (a JPEG file), it will be about 42 Kbytes. The inference is that the 
download times for these three equivalent files are 120 seconds, 60 seconds, and 10 
seconds, respectively. 

A number of lossy compression methods have been developed in which the transform 
compression has proven to be the most valuable. The best example of transform 
compression is embodied in the popular JPEG standard of image encoding. JPEG is 
named after its origin, the Joint Photographers Experts Group. Transform compression 
is based on the fact that when the signal is transformed, the resulting data values will 
be no longer equal to their information carrying roles. In particular, the low frequency 
components of a signal are more important than the high frequency components. Thus, 
removal of large number of bits from the high frequency components causes very small 
loss of the encoded information. 

Transform-based coding schemes achieve higher compression ratios for lossy com- 
pr^sion but suffer from blocking artifacts at high-compression ratios. Multiresolution 
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approaches are suited for lossy as well as for lossless compression. At lossy high- 
compression ratios, the typical artifact visible in the reconstructed images is the ring- 
ing effect. A comparative study of DCT and wavelet based image coding technique 
is available in [10]. New applications in a multimedia environment need some new 
functionalities of the image coding schemes. For that purpose, second-generation coding 
techniques segment the images into semantically meaningful pairs. Therefore, parts of 
these methods have been adopted to work for arbitrarily shaped regions. In order to 
add another functionality, such as progressive transmission of the information, specific 
quantization algorithms must be defined. 

Comparison of transform-based image compression methods with 
the prediction methods 

• Transform-based methods can better preserve the subjective image quality, and 
are less sensitive to statistical image property changes both inside a single image 
and between images. 

• Prediction methods, on the other hand, can achieve larger compression ratios in a 
much less expensive way, tend to be much faster than transform-based or vector 
quantization compression schemes, and can easily be realized in hardware. 

• If compressed images are transmitted, an important property is insensitivity to 
transmission channel noise. Transform-based techniques are significantly less sen- 
sitive to the channel noise - if a transform coefiicient is corrupted during trans- 
mission, the resulting image distortion is homogeneously spread throughout the 
image or image part and is not too disturbing. 

• Erroneous transmission of a difference value in prediction compression methods 
causes not only an error in a particular pixel but it also influences values in the 
neighborhood and it causes a considerable visual effect in a reconstructed image. 

This work is related to the Discrete Cosine Transform (DCT) based image compression 
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so various aspects of this scheme will be discussed in this Chapter. To point out the 
advantages and problems with this scheme, the block DCT and coding artifacts are 
discussed in the next section. 

1.2 Block Discrete Cosine Transform (BDCT) and 
Blocking Artifacts 

Discrete Cosine Transform (DCT) [11-18] has been widely used as a standard method 
for the compression and coding of image and video pictures. The standards specified by 
JPEG, MPEG and H.263 apply DCT for encoding the images before the compressed data 
is transmitted or stored. BDCT is the most widely used transform for the compression 
of both still and sequence of images. In DCT-based image compression, image is first 
divided into blocks of suitable size and then each block is transformed into DCT domain. 
In this scheme, high compression ratio is usually achieved by discarding information 
about the BDCT coefficients that are considered unimportant and yields image that 
exhibit the visually annoying blocking artifact. 

1.2.1 Block Discrete Cosine Transform 

Many transforms have been investigated for data compression; some of them have been 
invented specifically for this purpose. For instance, the Karhunen-Loeve transform 
(KLT) provides the best possible compression ratio, but is difficult to implement. The 
Basis functions of KLT are image dependent where as other transforms like Discrete 
Fourier Transform (DFT), Discrete Cosine Transform (DCT), Discrete Sine Transform 
(DST), and Discrete Hartley Transform (DHT) are image independent. The DFT is 
easy to use, but does not provide adequate compression. After much competition DCT, 
the relative of the Fourier transform, has been found to be the best. Just as the Fourier 
transform uses sine and cosine waves to represent a signal, the DCT only uses cosine 
waves. The 64 basis functions of DCT are shown in Fig. 1.1. DCT is a lossless, reversible 
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process, which can de-correlate the image signal and achieve energy concentration. It 
can therefore reduce the redundancy in the image signal before transmitting through a 
band-limited channel, or storing it in a memory-limited media. DCT is best suited for 
natural images. The mathematical definition of 2D DCT and IDCT for NxN block are 
given by: 


r/ ^ 2 Tr{2m + l)u) 7r(2n-M)u) ^ 

Fiu,v) = -^c(«)c(u)[2] Yu )cos( — )] (1.1) 

^ m=0 n=0 
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= ^[Y Y c{u)c{v)F{m,n) * (1.2) 


2N 


2N 


tx=0 t;=0 

where c(w) = 2~^/^ for w = 0; c{w) = 1 for w = 1, 2, Several desirable properties 

of DCT made it so popular for data compression: 


1. It exhibits very good energy compaction and de-correlation properties. DCT is 
real and orthogonal transform. Linear transform like DCT and KLT perform a de- 
correlation of incoming signal resulting in a compaction of the energy into small 
number of AC coefficients of lower order. These transform play an important 
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role when signal de-correlation is the major property required. KLT is optimum 
transform but difficult to compute efficiently. BDCT takes advantages of local 
spatial correlation properties of image. 

2. DCT is asymptotic approximation to the optimal KLT when the statistical prop- 
erties of image can be described by a first order Markov model. 

3. DCT can be computed very efficiently using fast algorithm similar to Fast Fourier 
Transform (FFT) algorithm. 

1.2.2 Blocking Artifacts 

In DCT based image compression scheme, the reconstructed image from highly com- 
pressed data has noticeable degradation due to blocking artifacts. In BDCT based 
image compression schemes, some high frequency DCT coefficients are transmitted with 
less accuracy. In this scheme, high compression ratios are usually achieved by discarding 
information about the BDCT coefficients that are considered to be unimportant. It is 
done by the quantization of DCT coefficients. This process is irreversible process that 
results in noticeable degradation near the block boundaries of decoded image. This 
visually annoying effect is called “blocking artifact”. The blocking artifact manifests 
itself as an artificial boundary among the pixels of adjacent blocks and constitutes a 
serious bottleneck for many important visual communication applications where the 
pleasing image is required at very high compression ratios. In case of very low bit 
rate coding (8 kbps to 64 kbps) compression ratio is also very high. Thus, the main 
objective of image compression is to get necessary bit rate with sufficient image quality 
and reasonable hardware cost. 

Blocking artifacts causes major degradation in block-based image coding techniques, 
especially at higher compression ratio. One main reason of blocking artifacts is that 
blocks are encoded and quantized independently without considering the correlation 
between adjacent blocks. Apart from blocking artifacts, ringing is another annoying 
artifact that appears frequently in transform based cjjmpressed image. The ringing 
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artifacts appear as sharp oscillations along the edges in compressed image. Ringing is 
the Gibb ’s phenomenon due to truncation of high frequency components by quantization. 
Effect of noise in edge area is less due to masking effect. These coding artifacts cause 
three types of noise: grid noise in monotone area, staircase noise near the edges and 
corner outliers at the cross point of M x M block. Noise in block coding is correlated 
with the local characteristics of signal. Grid noise in monotone area causing visually 
annoying artifacts manifests itself as an artificial boundary among the pixels of adjacent 
blocks. Staircase noise in edge area causes ringing which blurs the edges. 

In monotone areas where intensity changes gradually, there is tendency for intensity 
to change abruptly from one block to another. The abrupt changes in intensity are 
often visible since they occur along the block boundaries. An edge usually partitions the 
image into, two monotone areas. Block coder can maintain continuity of edges within the 
block but cannot assure this across the blocks. Therefore, edges tend to be unsmooth 
in block boundaries. 

In BDCT based image compression, the image degradation becomes visible when 
compression ratio exceeds certain level. These degradations manifest themselves as 
blocking artifacts due to the rigid block partitioning of the image and ringing noise 
around edges due to the course quantization. These blocks are processed independently 
but the DCT basis functions do not generally decay smoothly to zero. Thus, they 
exhibit discontinuities when thought of as being padded by zero beyond the boundaries. 
Therefore independent processing of spectra of adjacent blocks generates the well-known 
blocking artifacts. 

1.3 DCT Based Image Compression and Coding 

The DCT based image compression is discussed in detail in [19-21]. A typical DCT 
operates on blocks of image or video sequences of size 8x 8. Although DCT is a lossless 
process by itself, quantization of the DCT coefficients is a lossy process. The quantized 
DCT coefficients are scanned in a zigzag order, before the run-level coding is performed. 
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Through zigzag scanning and run-level coding, the numbers of zero coefficients preceding 
anon-zero DCT coefficient, except for the DC coefficient, are grouped together to form a 
RUN-LEVEL code. The run-level code can be encoded by fixed or variable-length codes 
(VLC), such as Huffman code. A replica of the original image can be reconstructed at 
the receiver by reversing the processes at the transmitter, i.e., VLC decoding, run-level 
decoding, inverse zigzag scan, scaling and inverse DCT. Simplified block diagram of 
DCT based image compression scheme is shown in Fig. 1.2. The DCT coefficients have 
an important property that their values usually decay rapidly as one scans through the 
block of image in a zigzag order. Most of the AC coefficients have relatively small values 
as compared to the DC coefficient, i.e., the first coefficient. Therefore, a considerable 
number of these small coefficients can become zero after quantization. The zigzag scan 
and run-level coding makes use of this property in order to lower the bit rate required to 
transmit the relevant information. In trade-off, the reconstructed image will be distorted 
in accordance with the characteristics of the quantizer. The main steps of DCT based 
compression scheme are: 

• Division of image into blocks 

• Block Discrete Cosine Transformation (BDCT) 

• Quantization 

• Zigzag Scan 

• DPCM on DC coefficient 

• RLE on AC coefficient 

• Entropy Coding 

These are also the major steps of the most popular JPEG image compression algorithm. 
The JPEG image compression algorithm provides a very effective way to compress images 
with minimal loss in quality. JPEG is a good example of how several data compression 
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Figure 1.2: Block Diagram of DCT Based Image Compression scheme, (a) Encoder (b) 
Decoder 
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schemes can be combined for greater effectiveness. This standard is discussed in detail 
in Appendix A. Now, the remaining steps will be discussed in brief in the following 
subsections. 

1.3.1 Quantization 

In BDCT based image compression, high compression ratios are usually achieved by 
discarding information about the BDCT coefficients by quantization process. In any 
real coding system, the representation of coefficients using finite number of bits requires 
the presence of quantizers. Even a non-orthogonal or sub optimal transform can achieve 
smaller mean square error compared to optimal KLT, in noisy case. There is a tradeoff 
between image quality and degree of quantization. A large quantization step size can 
produce unacceptably larger image distortion. Unfortunately, finer quantization leads 
to lower compression ratios. The question is how to quantize the DCT coefficients most 
efficiently. Because of human eyesight’s natural high frequency roll-off, these frequencies 
play a less important role than low frequencies. Thus, DCT based schemes uses a 
much higher step size for the high frequency coefficients, with little noticeable image 
deterioration. 

Quantization Table 

The quantization strategy in DCT based coding is expressed in the form of quantization 
table Q, which specifies the step size for mid-tread uniform scalar quantization of each 
coefficient. The quantization matrix is the 8x8 matrix of step sizes (one element for each 
DCT coefficient). Step sizes will be smaller in the upper left (low frequencies), and larger 
in the upper right (high frequencies); a step size of 1 is the most precise. The quantizer 
divides the DCT coefficient by its corresponding element in quantization matrix, and 
then rounds it to the nearest integer. Division by large value drives small coefficients 
down to zero. This results in many high frequency coefficients becoming zero, and 
therefore easier to code. The low frequency coefficients undergo only minor adjustment. 
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Figure 1.3: Zigzag Scanning. 


The most common approach for finding a quantization table for a desired rate/distortion 
target is to use default quantization table and scale it by a scalar multiplier (called the 
quality factor) until the target is achieved. Thus, multiplying the default quantization 
table by quality factor controls the quantization step size for desired compressed file size. 

1.3.2 Zigzag Scan 

After quantization, it is not unusual for more than half of the DCT coefficients to be 
equal to zero. JPEG incorporates run-length coding to take advantage of this. For each 
non-zero DCT coefficient, JPEG records the number of zeros that preceded the number, 
the number of bits needed to represent the numbers amplitude, and the amplitude itself. 
To consolidate the runs of zeros, JPEG processes DCT coefficients in the zigzag pattern 
as shown in Fig. 1.3. 

1.3.3 Differential Pulse Code Modulation (DPCM) on DC co- 
efficient 

Since the blocking artifact mainly from the DC coefficient is sensitive to spatial frequency 
response by human visual system, the DC coefficient is treated separately from the 63 
AC coefficients. DC coefficient is large and varied, but often close to previous value (like 
lossless JPEG). Therefore difference from previous 8x8 blocks is encoded using DPCM. 
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1.3.4 Run Length Encode (RLE) on AC coefficients 

After zigzag scan, the 1x64 vector has lots of zeros in it. The run length coding is used 
to encode it as (skip, value) pairs, where skip is the number of zeros and value is the 
next non-zero coefficient. 

1.3.5 Encoding 

The number of previous zeros and the bits needed for the current numbers ampfitude 
form a pair. Each pair has its own code word, assigned through a variable length code 
(for example Huffman, Shannon- Fano or Arithmetic coding). JPEG outputs the code 
word of the pair, and then the codeword for the coefficient’s amplitude (also from a 
variable length code). After each block, JPEG writes a unique end-of-block sequence to 
the output stream, and moves to the next block. When finished with all blocks, JPEG 
writes the end-of-file marker. 


1.4 Postprocessing of DCT coded images: State-of- 
the-Art 

Currently, transform-based image compression is very popular and finds various applica- 
tions. Especially, most of the international standards for image and video compression, 
such as JPEG, H.261, H.263 and MPEG recommend the use of block Discrete Cosine 
Transform (BDCT) as a main compression technique. The DCT scheme has potential to 
be used as a compression scheme because of the desirable energy compactness property 
and low computational complexity. 

In DCT based compression schemes, as discussed earlier, the high compression ratios 
are obtained by discarding information about DCT coefficients that is considered to be 
less important. The major drawback is visible discontinuities along block boundaries, 
commonly referred to as blocking artifacts. These often limit the maximum compression 
ratios that can be achieved. A number of techniques have been studied to reduce the 
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visual effects of coding artifacts. Coding artifact reduction method can be classified 
into two categories: First is variation in transform structure such as the interleaved 
block transform, the lapped transform and the combined transform and the second is 
postprocessing techniques. The postprocessing techniques are attractive because they 
keep codec itself unchanged. Thus one can use commercial products while implementing 
hardware systems. If we consider the coded images to be distorted by a codec system, 
we can apply image restoration techniques for the coding noise reduction. 

Postprocessing of DCT coded image is a very promising area for research. The 
performance of such system at high compression ratio is limited by blocking artifacts. 
The reconstructed image from highly compressed JPEG data has noticeable degradation 
due to blocking artifacts. Therefore the main objective of postprocessing is to remove 
this with lowest computational complexity. Linear, space-invariant filtering is inadequate 
to remove these artifacts. Iterative method such as Projection on to Convex Set (POCS) 
and Constrained Least Square (CLS), are not suitable for real time applications because 
of their greater computational complexity. So, space-variant or adaptive filtering is 
required. These schemes are based on edge information extracted from the received 
blocky image, which contains false edges due to blocking artifacts. Recovery of accuracy 
loss in DCT coefficient by discontinuity criterion is another method that can reduce 
these artifacts. It has low computational complexity and gives good results in monotone 
area. 


1.5 Literature Survey 

This work is focused on the postprocessing of image so literature survey on variation of 
transform structure is not being presented. Various postprocessing techniques have been 
published that reduce these blocking effects, but most of them introduce unnecessary 
blmaring, ringing, or other artifacts. In this section, a literature survey with brief 
description of various existing postprocessing techniques is presented. For this purpose, 
various postprocessing method can be divided into following categories: 
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. (i) Low Pass Filtering 

(ii) Space- variant/ Adaptive Filtering 

(iii) Regularized Image Recovery 

(iv) Block Boundary Discontinuity criterion 

(v) Prediction of Boundary Pixels 

(vi) Wavelet Transform 

1.5.1 Low Pass Filtering 

Considering the fact that artifacts are visible due to spatially high frequency components 
produced by the block discontinuity. Several kinds of lowpass filters are applied and it 
is observed that the Gaussian. Low Pass Filter (LPF) gives the best performance. The 
main draw back of lowpass filtering is that it smoothen out the image details such as 
edges. Therefore, in lowpass filtering, special care should be taken not to smoothen out 
the image details. Since block and ringing effects resulting from block DCT coding are 
high frequency artifacts, a straightforward solution is to apply LPF to the region where 
they occur. In this respect Reeves and Lim [22] in 1984 applied 3x3 Gaussian LPF 
only to those pixels along block boundary. 

1.5.2 Space- variant/ Adaptive Filtering 

The noise in block-coded images is correlated with the local characteristics of the signal; 
so space-invariant filters are unable to exploit the correlation to reduce the noise. These 
filters are not suitable because they will blur the edges and degrade the image visual 
quality. This is because most of the grid noise is out-of-band, while staircase noise 
is in the direction parallel to the edge. Signal bandwidth in monotone area is much 
lower than the bandwidth of the average spectrum of the edge blocks. Thus, this filter 
cannot remove locally out-of-band noise in monotone area. It is also found that the 
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human visual system (HVS) is more sensitive to blocking artifacts in monotone regions 
than edge regions. Thus, effect of noise is less visible in edge area than monotone 
area due to this masking effect of HVS. Therefore, a strong smoothing filter is required 
in monotone area whereas in edge regions the smoothing operation tend to introduce 
undesirable blur, so smoothing of a few pixels around block boundaries by directional 
lowpass filter is sufficient. Hence, space-variant or adaptive filtering depending on local 
image characteristics is preferable. Thus classification of image regions is used in these 
schemes to capture the non-stationary behavior of images. The goal of classification is to 
classify the image into different regions based on their statistical properties such that an 
optimum filters can be applied in each of the region. The information about local image 
characteristics can be extracted from image by edge detection or measuring flatness of 
the region. Thus, one of the best approach of reducing blocking artifact is to classify the 
image blocks into various categories and each is processed by one or two-dimensional 
filter. Several postprocessing algorithms based on space- variant/ adaptive filtering have 
been proposed to reduce blocking effect of block coding system [1-3,23-28]. 

In adaptive filtering, the coefficients of 2D filter are chosen by edge information of 
the image. Generally, no filtering is done at edge pixels but weighted or unweighted 
filtering is performed at the pixels in the monotone area or near the edge. 

Both non-linear space- variant filtering and adaptive filtering requires the classifica- 
tion of image, which is based on available edge information extracted from the received 
blocky image. 

Ramamurthi and Gersho [1] in 1986, proposed a non-linear space-variant filtering 
to reduce both so-called “staircase noise” and “grid noise” . In this algorithm, the image 
is divided into monotone area and edge area using edge information obtained from 
deteeting edge by gradient operator. Here two-dimensional lowpass filter is used to 
reduce grid noise in monotone area, whereas a directional one-dimensional filter is used 
to reduce staircase noise along the edges. The main features of their approach are: 


• Space-variant filter reduces locally out of band noise. It is found that most of the 



Chap. 1 Introduction 


16 


grid noise is out of band while staircase noise is out of band only in the direction 
parallel to the edge. Therefore, in this algorithm, they used separable 2D 3 x 3 
FIR filter to reduce blocking artifacts in monotone area and ID FIR filter was 
applied in the direction parallel to the edges to reduce the staircase noise in the 
area containing the edges. 

• Signal bandwidth in monotone regions is much lower than the bandwidth of average 
spectrum of edge block. 

• All ID filters are designed using Parks-McClellan algorithm. Bandwidth of the 
monotone area is computed from average spectrum of 8 x 8 block in the monotone 
area. 

• Edge orientation is found by gradient thresholding algorithm. Here the ratio of 
change in intensity to the average intensity is compared with a threshold. Two 
counter K and L are set to zero. For horizontal adjacent pixel counter K is 
incremented for positive gradient above preselected threshold and it is decremented 
for negative gradient above threshold. Similarly counter L computed for vertical 
adjacent pixels. 

Monotone : if \ K \< m and | L |< m 
Edge, 0° : if \ K \< m and | L ]> m 

Edge, 45® : if \ K \>m and | L |> m, and sgn(K)=sgn(L) 

Edge, 90° : if \ K \> m and | L |< m 

Edge, 135° : if \ K \> m and | L |> m, and sgn{K) = —sgn{L) 

where m is some predetermined threshold, and sgn{.) is the usual signum function. 

Hsu and Chen: 1993 [23] in 1993, reported an adaptive scheme transforming 
the median filter progressively to the LPF as the filter approached the block boundary. 
Post-filtering (spatial or temporal) has advantage of a low computational complexity 
and adaptation to local statistics relies on good classification scheme. 
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Kuo and Hsieh [24] in 1995, proposed an adaptive postprocessor for reducing 
blocking artifacts in which the adaptation is achieved by changing filter coefficient 
according to the local characteristics of image and the blocking effects. Here a space- 
variant LPF is used to smooth the pixel at block boundaries where the blocking artifacts 
are highly visible. In addition, the pixels that are close to an edge are adaptively filtered 
by directional lowpass filter. The edge pixels are left untouched by the postprocessor. 
Therefore, blocking artifacts can be removed without blurring the edges. The positions 
of edges are detected and a 2D regional or directional LPF is specially designed to remove 
blocking artifacts without blurring edges. 

Hsia et al. [2] in 1997, proposed transform domain classification and algorithm for 
blocky strength (sum of absolute value of A.C. components) estimation. This transform 
domain algorithm has low computational complexity. Adaptive FIR filter is used to 
remove blocking artifacts. Stronger filter is used for greater blocky strength (results in 
larger energy spread) where as narrow filter is used for smaller blocky effect. 

Castagno et al. [25] in 1998, applied rational filter for reducing blocking artifacts. 
The basic concept of the rational filter, which was devised to perform edge-preserving 
noise smoothing, is to modulate the coefficients of linear lowpass filter in order to limit 
its action in presence of image details. Its input/output relation is the ratio of two 
polynomials. Roughly speaking, the numerator has lowpass behavior, while the denom- 
inator is a function of difference between couples of pixels within the filtering mask. 
For large difference, it is assumed that the mask is located across a signal transition, 
and frequency response of the lo^wpass filter is automatically made less selective in the 
direction of the signal transition itself. 

Lee et al. [26] in 1998, proposed a algorithm for removing coding artifacts in JPEG 
compressed image. In this algorithm the blocks are classified according to type of noise: 
grid noise, staircase noise, and corner outliers. It makes use of signal adaptive filter 
based on edge information as well as comer outliers detection and replacement scheme. 
Here Sobel’s operator is used to find edge map which has advantage of differentiating and 
smoothing along the orthogonal directions. A signal adaptive filter performs weighted 
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or unweighted filtering depending on whether edge points are included or not included 
in 5 X 5 filter window. 

Park and Lee [3] in 1999, proposed a method for reducing quantization effects 
in highly compressed MPEG data. The reconstructed image have degradations, like 
blocking artifacts near the block boundaries, corner outliers at cross points of blocks, 
and ringing noise near the edges. It reduces these artifacts adaptively by using both 
spatial and temporal information extracted from the compressed data. The blocking 
artifacts are reduced by ID horizontal and vertical LPF whereas the ringing noise is 
reduced by 2D signal adaptive filter (SAP). The blocking and ringing semaphores of 
each block greatly contribute to reducing computational complexity. In this method, the 
corner outliers are detected and compensated. This algorithm has low computational 
complexity. 

Kim et al. [27] in 1999, proposed a de-blocking filter with two separate modes 
in low bit rate block based video coding. The first mode corresponds to flat regions 
where a strong ID filter is applied because flat regions are more sensitive to the human 
visual system (HVS). In the second mode corresponding to other regions, a sophisticated 
smoothing filter based on the frequency information around block boimdaries is used to 
reduce blocking artifacts adaptively without introducing blur. 

Apostolopoulos and Jayant [28] in 1999, proposed a algorithm in which pixel 
by pixel processing is used to identify and reduce blocking artifacts while preserving 
sharpness and naturalness. In this algorithm artifacts are detected just by counting 
number of coefficients used to represent the block. The advantages of this simple scheme 
are the robustness and the low computational complexity. The algorithm is based on 
the assumption that surrounding pixels are less distorted and can improve estimate 
of distorted pixels. Therefore the filter is applied along the boundaries but updating 
the pixels within the distorted block. To reduce mosquito noise, the edge pixels are 
carefully identified and preserved while remaining non-edge pixels are smoothened. They 
have used a pixel based edge detection algorithm. To avoid false edge detection due to 
mosquito noise it is essential to distinguish between true and false edges. To counteract 
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this problem, their approach is based on idea that a true edge has a string of connected 
pixels, while false edges are isolated points. 

Comes at el. [29] in 1999, proposed a scheme for the restoration of the visual quality 
of image affected by coding noise. It is based on the tuned channels model of human 
perception. It uses the phenomenon of masking as criterion for the processing of the 
noise. Their aim of postprocessing was to improve the image quality by processing the 
perceptual channels of the noisy image where the noise is above visibility threshold. This 
operation is called filtering of unmasked noise. Postprocessing is based on a model of 
the human visual system that considers the relationship between visual stimuli and their 
visibility. This phenomenon is known as masking which is used as a criterion for the 
locally adaptive filter design. An image transformation that yields visual stimuli tuned 
to frequency and orientation according to the perceptual model is proposed. It allows 
a local measure of the masking of each perceptual stimulus considering the contrast 
between signal and estimated noise. This measure is obtained by analytic filtering. 
Processing schemes are presented with applications to discrete cosine transform (DCT) 
and subband coded images. 

Paek at el. [4] in 2000, proposed a DCT based postprocessing method to alleviate 
blocking artifacts in transform coded images. The high frequency components are mainly 
caused by edge components. In this approach, two adjacent blocks are considered as 
homogeneous if no adjaeent pixel difference is greater than the difference of block- 
boundary. Then a local frequency characteristic in homogeneous block is examined 
through DCT. They also derived the relationship between the DCT coefficients of two 
homogeneous blocks of different sizes. By considering the information about the original 
edge and using this relationship, they detected the high frequency components due to 
blocking artifacts. By forcing their high frequency components to zero, they reduced 
the blocking artifacts in two homogeneous blocks. 



Chap. 1 Introductioa 


20 


1.5.3 Regularized Image Recovery 

Basic idea is to impose a number of constraints on the coded image in such a way as 
to restore it to its original artifact free form. One such constraint can be derived by 
exploiting the fact that the transform coded image suffering blocking effects contains 
high frequency vertical and horizontal artifacts corresponding to vertical and horizontal 
discontinuities across boundaries of neighboring blocks. Since these components are 
missing in the original uncoded image, one step of iterative procedure consists of pro- 
jecting the coded image onto the set of signals that are band limited in the horizontal 
or vertical directions. This constraint is referred as filtering constraint. Two similar 
approaches based on regularized image recovery are Projection onto Convex Set (POCS) 
and Constrained Least Square (CLS). Both the approaches are almost free from blocking 
artifacts but main drawback is their computational complexity. These methods need 
significant increase of processing time due to iterative procedure. So they are not suitable 
for real time video processing. These schemes will be discussed in brief in the following 
subsections; 

Projection onto Convex Set (POCS) 

The restoration algorithm based on POCS has advantage that it enables a large number 
of a priori (desirable) known constraints to be incorporated in the algorithm through 
the mechanism of POCS, if the constraints can be associated with convex set. If the 
convex sets have common iatersection, the sequence of iteration converges to a restored 
image that satisfies aU of the constraints. For POCS method, a new constrained set 
is defined that captures the smoothness between the blocks. The convexity of the set 
shown and projection on to it is computed. Thus, the reconstructed image is obtained by 
altering projection onto the smoothness set and the set defined by the available partial 
information about the BDCT coefl&cients. 
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Zakhor [30] in 1992, first tried to apply the Theory of POCS to the reduction of 
blocking artifacts of block coded images and suggested an iterative algorithm. Alternat- 
ing projection onto two sets, starting from blocky image, reduces the blocking artifacts. 
The quantization constraint defines a set containing all images that are mapped by the 
video coder onto the same blocky image. This is accomplished by the set of bandlimited 
images obtained by lowpass filtering. While first set enforces the restored image to be 
close to the received blocky image, the projection on to second one carries out the actual 
enhancement. Blocking artifacts are reduced but at the cost of some blurring. Reeves 
and Eddins pointed out in [31] that the theory of POCS is inadequate as a basis 
for justifying the algorithm with the lowpass filter employed. This is mainly because 
non-ideal lowpass filter is not a projection operator onto a convex set. 

Yang-et al. [32] in 1993, proposed three constraint sets based on the theory of 
POCS. The first is the quantization constraint set (QCS) whereas other two sets are 
smoothness constraint sets (SCS), contains information that is lost. The main features 
of POCS based approach are: 

• Every known property of the original image (available data and prior knowledge) 
is represented by closed convex set. 

• A vector common to all the sets can be found by alternating the projection onto 
each one of them, starting from any initial guess vector. 

• Filtering constraint: available transform coded image contains high frequency 
vertical and horizontal artifacts corresponding to vertical and horizontal disconti- 
nuities. 

• Quantization constraint: captures the between block smoothness 
Constrained Least Square (GLS) 

In CLS method, an objective function is introduced that incorporates into the recovery 
process both requirements that is fidelity to the available information about the BDCT 
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coefficients and smoothness. Objective function which consists of two terms, is itera- 
tively minimized. The first term measures the blockiness of an image by applying a 
high pass operator and is minimized for smooth images. The second term calculates 
the deviation from received blocky image. The regularization operator captures the 
smoothness properties of the image. The tradeoff between the conflicting requirements 
of the fidelity to the available data and smoothness is controlled by regularization pa- 
rameters. Thus, minimization of objective functions reflects the smoothness properties. 
To incorporate quantization constraint, the solution after every iteration is projected 
onto the corresponding set of images. 

Yang at el. [32] in 1993, used this method for reducing blocking artifacts, which is 
similar to POCS method. The recovered image is obtained by minimizing an objective 
function that imposes conflicting requirements on the reconstructed image. In this 
approach, if one of these function penalizes deviation from the available data, the other 
must penalize the undesired effects if the image is reconstructed only from the available 
data or, constraints the behavior of the reconstructed image. Reconstructed image is 
almost free from blocking artifacts butt computational complexity of this method is 
more due to its iterative nature 

Minami and Zakhor [33] in 1995, proposed an optimization approach for removing 
blocking effects in transform coding. This technique exploits the correlation between 
the intensity values of boundary pixels of two neighboring blocks. It is based on the 
theoretical and empirical observation that under mild assumption, quantization of the 
DCT coefficients of two neighboring blocks increases the expected value of mean squared 
difference of slope (MSDS) between the slope across two adjacent blocks, and the average 
between the boundary slops of each of the two blocks. The amount of this increase is 
dependent upon the width of the quantization intervals of the transform coefficients. 
Therefore, among all permissible inverse quantized coefficients, the set which reduces 
the expected value of this MSDS by appropriate amount, is most likely to decease 
the blocking artifacts. To estimate the set of unquantized coefficients, they solved 
constrained programming problem in which the quantization decision intervals provide 
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upper and lower bound constraints on the coefficients. The approach is based on gradient 
projection method. 

Morkov random field (MRF) have been successfully used to model images. Their 
generality for modeling and capacity to incorporate different constraint and character- 
istics e.g. smooth region, edges etc., have made them popular. They have been used to 
develop algorithm to deconvolve blur and remove white noise. O’Rourke and Steven- 
son [34] in 1995, proposed a method for producing higher quality reconstructed image 
based on a stochastic model for the image data. Quantization partitions the transform 
coefficient space and maps all points in partition cell to representative reconstruction 
point, usually taken as the centroid of the cell. The proposed image restoration technique 
selects the reconstruction point within the quantization partition cell which results in a 
reconstructed image that best fits a non-Gaussian Morkov random field (MRF) image 
model. This approach results in a convex constraint optimization problem that can be 
solved iteratively. 

Ozcelik et al. [35] proposed an iterative technique to reduce the unwanted degrada- 
tion such as blocking and mosquito artifacts while keeping the necessary detail present in 
the original image. The proposed technique makes use of a priori information about the 
original image through a non-stationary Gauss-Markov model. Utilizing this model, o 
maximum a posteriori (MAP) estimate is obtained iteratively using mean filed annealing 
(MFA), which is an optimization technique. At each iteration, projecting the image 
onto constraint set defined by the quantization preserves the fidelity to the data. The 
maximization of the posterior function is carried out using man field annealing. Image 
models based on MRF have advantage that the field of interest (i.e. the intensity 
or motion) can be considered as consists of two fields: one representing the function 
that is to be estimated, and the other representing discontinuities. Furthermore, the 
probability distribution of the fields is characterized by a Gibbs distribution. The model 
containing the a priori information is specified through the “energy function” of the 
Gibbs distribution. The interaction of the variables characterizing image processing 
system is highly localized and the probability distribution of the fields is characterized 
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itribution. In such type of systems, the optimization using MFA closely 
;he optimal solution. 

al. [36] in 1999, proposed the application of hierarchical Bayesian paradigm 
istoration problem. In 2000, they also used the same Bayesian approach 
ion and transmission of regularization parameters for reducing blocking 
In this paper they proposed the application of Bayesian paradigm to 
BDCT compressed image and the estimation of the required parameters. 


Lestoration based on Adaptive Constrained Optimization. 

c [38] in 1998 have proposed a fast adaptive image restoration filter using 
je classification for reducing artifacts. This scheme can be summarized 
I steps: 


e degradation operator H = C~^D~^QC where [C] and [C]"^ are the 
,nd inverse DCT matrices respectively and [Q] and [i^]“^ are the quanti- 
d inverse quantization matrices respectively, then y = Hx. Degradation 
H results in both discontinuities at block boundaries and loss of high 
details inside blocks. Based on image restoration theory, the original 
s best estimated by applying the inverse of H to the blocky image y, if 
e of H exists and well defined. 


ive Constrained least square (CLS) filter along the direction of edge is 
iltering corresponding block. It incorporates a priori constraints into the 
n process without significant computational overhead. The frequency 
)f CLS restoration filter is given by 


G{k, 1) 


\H(k,l) P+A|A(i,0 P 


k, 1) and A(k, 1) are the 2D DFT of degradation operator and a high pass 
denominator oi G(k,l), | H{k,l) p tries to reduce the degradation due 
{k, 1) p suppresses excess amplification of high frequency components. 
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and the Lagrange multiplier A is determines such that the solution satisfies the 
1 1 Ax IP = subjected to above equation. 

• Find X such that ||y — jH’x|| = 0 subjected to |1 Ax|| < where A is high pass filter 
(HPF). 

• Hence the solution is such that which minimizes the variance across block boundary 
less than specified. 

• Two DCT coefficients C(0,1) and C(1,0) is used for edge classification. 

• Real time restoration filter is implemented as truncated FIR filter. 

Paek at el. [39] in 1998, proposed a scheme based on POCS to reduce the blocking 
artifacts. They assumed the original image is highly correlated so the global frequency 
characteristics of two adjacent blocks are similar to local ones in each block. They 
considered the high frequency components in the global characteristics of a decoded 
image, which are not found in the local ones, as the result of blocking artifacts. They 
employed N-point DCT to obtain local characteristics, and 2N-point DCT to obtain 
global ones, and derived the relation between N-point and 2N-point DCT coefficients. A 
careful comparison of N-point and 2N-point DCT coefficients makes possible to detect 
the undesirable high frequency components due to blocking artifacts. They have also 
proposed novel convex set and their projection operators in DCT domain. 

1.5.4 Block Boundary Discontinuity Criterion 

Tien and Hung [40] in 1993, proposed a method of adjusting the quantized transform 
coefficients of the decoded image. It first decides whether the block is over edges or in a 
monotone area. In monotone area, the DC and first order horizontal and vertical DCT 
coefficients are adjusted to reduce the discontinuity over block boundary. 

Error concealment problems in ATM network environment was handled by Wang 
c# el [41] to recover DCT coefficients lost during network transmission by maximally 
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smoothing the transition of pixel values over block boundary. Blocking artifacts reduc- 
tion can similarly be considered as an attempt to recover the accuracy loss of quantization 
coeflBicients. 

Jeon and Jeong [5] in 1998, proposed a postprocessing approach, which can achieve 
minimum discontinuity of pixel value over block boundaries by compensating the loss 
of coefficient accuracy in transform domain. To quantify blocking artifacts, the block 
boundary discontinuity measure is obtained as the sum of squared pixel differences over 
the four block boundaries. The blocking artifacts are caused by loss of accuracy of 
transform coefficients in the processes of independent quantization of each block. Since 
quantization takes place in transform domain, the effect of quantization error is spread 
over all of the spatial locations within the block. The Artifacts are more noticeable 
in plain or slowing varying areas. This method finds the compensating transform 
coefficient whose inverse transform can minimize the block discontinuity when added 
to the quantized received image. 

1.5.5 Prediction of Boundary Pixels 

Choy at el. [42] in 1997, proposed a technique based on the weighted least squares 
estimation of transform coefficients from their quantized version and the available infor- 
mation about the quantizers used. 

Chan at el. [43] in 1998, proposed a real time, non-iterative method to restore the 
block encoded images. This method is based on concept of linear prediction and vector 
quantization. Since the blocking artifact is a consequence of ignoring the inter-block 
correlation during compression therefore one of the best way to restore the original image 
is to make use of residual inter-block correlation to recover the block boundary pixels. 
Here the relation between two are investigated and then make use of the statistics obtain 
to predict the original pixel intensity. As considerable amount of residual inter-block 
correlation exists, the linear predication can provide a good prediction. In this approach, 
the election of prediction filter coefficients is based on the least squares minimization 
criterion. 
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L.5.6 Wavelet Transform 

[n [44-47] the blocking artifacts are reduced by using a wavelet transform. Xiong at el. 
44] in 1997, proposed a algorithm to reduce blocking artifacts using wavelet transform. 
By exploiting cross-scale correlation among wavelet coefficients, edge information in 
JPEG image is extracted and protected, while blocky noise in smooth background regions 
is smooth out in wavelet domain. This scheme has much lower computational complexity 
and able to achieve same PSNR as the best iterative method. In over complete wavelet 
representation signal is characterized by local maxima of wavelet transform modulus 
and finding these local maxima is equivalent to canny edge detector. Here, instead of 
finding the exact local maxima of wavelet transform modulus, cross-scale correlation is 
exploited for edge detection by directly multiplying the over complete representations 
of JPEG compressed image at diflEerent scales. Locations with cross-scale correlation 
above certain threshold are identified as edges, while low cross correlation regions are 
treated as background. So, by this method edges are extracted and protected across 
block boundaries. It is found that quantization noise is structured and blockiness has 
strong showings in first scale high-pass wavelet image. With edge information, high-pass 
wavelet image is thresholded to smooth out blocky discontinuities in background regions. 

Kim at el. [45] in 1998, proposed a method the blocking components for the signal are 
removed because they causes the stepwise discontinuities in the spatial domain and which 
appears as impulse at each block boundary in the first scale of the wavelet transform. 
For multiscale analysis, they used the wavelet transform for the characterization of edges 
and singularities in multiscales by Mallat and Zhong. Each discontinuity is classified into 
three classes, and the removal of discontinuities in multiscales is applied to each class in 
a different way. 

Choi and Kim [47] in 2000, proposed a scheme for postprocessing in wavelet 
transform domain in which they divided the blocking artifacts into two categories. 
One is blocky noise which exhibits a block pattern, and the other is granular noise 
which can be assumed white Gaussian. Subband decomposition makes the blocky noise 
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more noticeable as vertical and horizontal false edges periodically appearing at block 
boundaries in high frequency subbands. By identifying such high frequency components, 
they reduced the blocky noise with linear minimum mean square error filter. The 
granular noise can be reduced by the wavelet based thresholding method. 


1.6 Introduction to Digital Notch Filter 

The notch filter highly attenuates a particular frequency component in the input signal 
while leaving nearby frequency components relatively unchanged. One application of 
notch filter is in the elimination of sinusoidal interferences corrupting signal, such as 
power line interferences in the design of digital instrumentation systems. 

1.6.1 ID Digital Notch Filter 

Hirano et al. [48], have given the systematic design approach for designing the ID digital 
notch filter. Second order digital notch filter is characterized by two parameters, notch 
frequency and 3 dB rejection bandwidth. In the approach, they suggested the realization 
of notch filter by minimum number (two) of multipliers. The transfer function of analog 
notch filter is: 




s^ + 
bs + c^ 


(1.3) 


Here a; = c is the notch frequency at which no signal is transmitted through the filter. 
Within the fi:equency band centered at a; = c and of width b, all signal components are 
attenuated by more than 3 dB. Hence 3 dB rejection bandwidth is b. Digital notch filter 
can be developed by applying bilinear transformation's — [z — I)/ {z 1)) to analog 
notch filter. Using this transformation, the transfer function of digital notch filter can 
be obtained as: 

2;) = 1 + 


1 — a\z~^ + a2Z~^ 


(1.4) 
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where a\ and 02 in terms of notch frequency Uo and 3 dB rejection bandwidth BW 
are given by: 


_ 2cosci;o _ 1 — tan(BM^/2) 

“ 1 + tan(BW/2) “ 1 + tan(BW/2) 


1.6.2 ID HR Multiple-notch Filter 


Pei and Tseng [7] in 1997, have given a method for designing ID HR multiple- 
notch filter from allpass filter. In this method the specifications of notch filter is first 
transformed into that of allpass filter. Therefore, the notch filter design problem becomes 
an allpass filter design problem. The specifications of multiple-notch filter is given by: 




0 foru: = LONk k = l,2,...M 

1 otherwise 


(1.5) 


The transfer function of 2M-order allpass filter is given by: 

tt2M “I" d" 0,iZ z 


A{z) = 


1 -f aiz~^ -i- .... + a2MZ~^^ 


( 1 . 6 ) 


where and H{eP‘^)=^[l -(- To design allpass filter, a set of linear 

equation in the filter coefficient is generated from its phase characteristics and solved. 
The set of linear equation is given by 


Q.a = p (1.7) 

where 

a = [oi, 02 , ■■■( 12 mY 
p = [tanPi,tanP2, ■..tan/32MV 
Qik = sin{kwi) — tan^iCos{k(jJi) 

Thus the realization of notch filter H(z) is equivalent to realization of APF A(z). The 
H(z) is expressed as 


H(z) = [1 + A{z))]l2 


( 1 . 8 ) 
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Due to mirror image symmetry relation between the numerator and denominator poly- 
nomials of allpass filter, the notch filter can be realized by computationally efficient 
lattice structure. 

Joshi and Dutta Roy [49] in 1999, have modified the method for designing ID 
IIR-multiple notch filter. They realized all-pass filter is by cascade of N second order 
allpass filter. This method ensures those 3 dB rejection bandwidths are lower than those 
specified. 

1.6.3 Linear-phase FIR notch Filter 

To provide the linear phase, the digital notch filter must be a finite-duration impulse- 
response filter with a symmetric impulse response. Design of FIR digital notch filter has 
been discussed in [9, 50]. Yu et at [9] investigated some approaches for designing ID 
linear-phase FIR notch filters, which are based on the modification of several established 
design techniques of linear phase FIR band selective filters. In addition, the design of 
2D linear-phase FIR single notch filter is briefly considered. They exploited the three 
commonly used design techniques for designing linear phase FIR notch (LPFN) filter: 
(i) windowed Fourier series approach (ii) frequency sampling approach and, (iii) optimal 
filter design approach, based on Remez exdiange algorithm. They also considered the 
use of some special techniques for the linear phase FIR filter design, such as the frequency 
response sharpening approach and interpolated FIR filter technique. In windowed 
Fourier series approach for designing LPFN filter Kaiser window is used, which is also 
the most frequently used window for designing the FIR filter. 

1.6.4 2D Single Notch Filter 

Pei and Tseng [6,51] in 1994, developed a simple algebraic method for 2D HR filter 
design to yield not only closed form transfer function but also to satisfy bounded- 
input /bounded-output (BIBO) stability condition. They have applied this filter to elim- 
inate sinusoidal/narrowband interferences superimposed on an image. The frequency 



31 


Chap. 1 Introduction 

specifications of a 2D ideal notch filter is given by 

y 1 otherwise 

If Hp{zi,Z 2 ) is 2D-parallel line filter and Hs(zi,Z 2 ) is 2D-straight line filter, then desired 
notch filter is given by 


H^{Zi,Z2) = 1 - Hp{Zi,Z2)HsiZi, Z2) ( 1 - 10 ) 

To design these two filter they started with the frequency specifications of a 2D parallel 
line filter which is given by 

= | ^ for ^2 = ±oj 2 N 

[ 0 otherwise 

The 2D parallel line filters can be designed by choosing Hp{zi,Z 2 ) as: 

Hp{zuZ 2 )=Htpiz)l=,, for i = l,2 (1.12) 

where Hi,p{z) is ID-bandpass filter and given as: 


z ^ — aiz ^ -f- 02 
1 — aiz~^ + 02 ^;“^ 

In above expression oi and 02 are given by: 



(1.13) 


_ 2cosaJ*r _ 1 — tan{Biy/2) 

~ 1 + tan(BW/2) “ 1 -f- tan(BW/2) 

where Up is the center frequency and BW is the 3 dB bandwidth of Hf^{z). 2D straight- 
line filter is designed by frequency transformation of 2D analog protype filter. 

Pei and Tseng in 1997 [52] and in 1998 [53] reduced the 2D notch filter design 
problem into two pairs of ID filter design problem. In [52], they designed 2D HR notch 
filter by writing outer product expansion of sampled frequency response matrix of ideal 
2D notch filter using singular value decomposition. They also proposed similar approach 
to d^ign 2D FIR notch filter by singular value decomposition in [53]. 
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Huang et al. [8] have used notch filter in Fourier domain to remove the artificial 
edges due to blocking artifacts. In this scheme, the edge image is notch filtered and 
used for a neighborhood analysis. Then, a space-variant filter is applied to the coded 
image adaptively, dependent on frequency analysis. Blocking artifacts are expected 
to have periodic texture. According to signal processing theory, the periodic textures 
will results in some peak in the Fourier transform domain. This means the periodic 
texture can be detected more efficiently in frequency domain. In this approach, based 
on an analysis in frequency domain, the edge image is notch filtered and then used for a 
neighborhood analysis. A space-variant filter is applied to the coded image adaptively, 
dependent on frequency analysis. 

1.7 Summary of few algorithms proposed in this 
work 

A number of algorithms for reducing blocking artifacts are proposed in present work. 
First the hybrid scheme is proposed which takes advantages of adaptive filtering and the 
recovery method for the loss of accuracy of DCT coefficients. Then both HR and FIR 
2D multiple-notch filters are designed and applied to improve the performance of the 
space- variant/adaptive filtering method. At the end of this work few computationally 
efficient algorithms are also proposed. 

1.7.1 Hybrid Scheme 

From the available literature, it has been seen that the filtering of decoded image without 
considering the local image statistics often cause the loss of high frequency detail such as 
edges. Any single algorithm proposed in literature is not able to perform best in different 
types of regions of an image. Thus, a hybrid scheme is developed for postprocessing of 
DCT coded images by clubbing the merits of several existing algorithms. This scheme 
is named as hybrid scheme as the processing is done in both DCT as well as spatial 
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domain by two different algorithms. Smoothing artificial discontinuities due to blocking 
artifacts improves the image quality whereas smoothing actual image edges degrades it. 
Thus, objective was to satisfy these two conflicting requirements as simply as possible. 

Classification is essential in postprocessing which attempts to exploit local statistics 
of image regions and the sensitivity of human eyes. In this approach, the edges of 
BDCT coded image is detected and used for the classification of regions into monotone 
area and edge area. The scheme makes use of adaptive filtering in edge area and 
block-boundary discontinuity minimization algorithm in monotone area. For monotone 
area, the minimum discontinuity of pixel values over block boundaries is achieved by 
compensating for the loss of accuracy of transform coefficients by the algorithm suggested 
by Jeon and Jeong in [5] with some modification. For edge area, a signal adaptive filter is 
proposed which perform the filtering approximately parallel to the edge to avoid blurring. 
The use of a lower order signal adaptive filter and the block-boundary discontinuity 
minimization algorithm reduces the processing time. Corner outliers reduction scheme 
further improves the performance. 

It has been observed that the performance of this algorithm is better than space- 
variant filtering as well as the two individual algorithms used in hybrid scheme. It has 
been found that the performance of the proposed algorithm improves as compression 
ratio increases. The reduction of discontinuity signifies that the blocking artifacts are 
reduced. The improvement in PSNR for block-boundary pixels is more as compared to 
over all PSNR which also indicates the same. 

1.7.2 Design and applications of 2D HR multiple-notch filter 

In present work, the design and some new applications of 2D HR multiple-notch filter 
have also been proposed. In proposed design approach, a simple algebraic method [6] is 
used to design the 2D HR multiple-notch filter from ID multiple tone filter. The design 
of ID multiple tone filter is based on the design of all-pass filter as discussed in [7]. 

The possibility of using 2D multiple-notch filter for reducing blocking artifact is fully 
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examined. It is shown that 2D multiple-notch filter can eliminate the square grid. It 
can also reduce the blocking artifacts from DCT coded images. The concept of reducing 
blocking artifacts by 2D multiple-notch filter is based on the fact that the discontinuities 
due to block artifacts in decoded image have periodic structure which results in some 
peaks in DCT domain. These discontinuities are highlighted in gradient of the image. A 
2D multiple-notch filter can kill these peaks to remove the false edges from the edge-map 
of blocky image. Discontinuities due to blocking artifacts are more visible in monotone 
area, so this filter can be applied directly in these areas. Thus, improvement in the 
performances of space- variant/adaptive filter by removing of false edges from the edge- 
map and the reduction of blocking artifacts directly in monotone area, are the new 
possible applications of 2D multiple-notch filter. In these applications, the reduction of 
discontinuity signifies that the blocking artifacts are reduced. 

1.7.3 Design and applications of 2D FIR multiple-notch filter 

The design approach of 2D multiple-notch filter has also been extended to FIR type. 
FIR filters are suitable for image processing applications as they cause less distortion due 
to their linear phase property. The drawback of FIR filter is that they cause more delay. 
A special 2D FIR multiple-notch filter is designed with an intention to remove blocking 
artifacts from DCT coded images. The design procedure for the 2D FIR multiple-tone 
filter is similar to HR type of this filter but the desired 2D multiple-notch filter is the 
complementary filter of 2D multiple-tone filter. In this case, two ID FIR multiple-tone 
filters are designed by windowed Fourier series method with Kaiser window. This filter 
is also applied for the reduction of blocking artifacts and removal of false edges from 
edge of blocky decompressed image. The results obtained in this case are better than 
HR type of this filter as expected. 
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1.7.4 Smoothing of the Sub-image of DC Coefficients by Signal 
Adaptive Filter 

The algorithm is based on the assumption that the correlation among the DC coefficients 
of neighboring block is high if all the blocks belong to monotone area. This algorithm 
reduces the blocking artifacts by exploiting the residual inter-block correlation which 
exists in received blocky image. In the approach, a DC sub-image is constructed by 
taking only the DC coefficient of each block in DCT coded image. A signal adaptive filter 
is applied to smooth the DC sub-image. The image blocks are classified as monotone 
block or edge block simply by comparing the absolute value of sum of the AC coefficient 
of the block with a threshold. This scheme shows significant reduction in blocking 
artifacts without much computational complexity. This can be verified by calculating 
block boundary discontinuity. The PSNR is slightly improved but the improvement is 
more at block boundary pixels. 

1.7.5 Reduction of Block-boundary Discontinuities 

Proposed algorithm is based on the assumption that artificial discontinuities at block 
boundaries are due to blocking artifacts and are more visible in monotone or slowly 
varying areas. The algorithm reduces the discontinuities in blocky image by capturing 
and removing the discontinuities in DCT domain. The algorithm also reduces the 
blocking artifacts significantly by selective attenuation of AC components corresponding 
to block boundary discontinuities. In order to capture the artificial discontinuities, the 
blocky image is re-divided into blocks with different block boundaries. To reduce the 
artificial discontinuities, the selected AC components of those new blocks which do not 
contain details, are heavily attenuated. The algorithm requires the computation of DCT 
of new blocks but the processing time is less because no filtering is required. Availability 
of fast algorithms for DCT computation further reduces the processing time. 
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The above two algorithms together reduce the blocking artifacts considerably. It is 
verified by comparing boundary PSNR and block-boundary discontinuity. It is found 
that improvement in PSNR for block boundary pixels is more as compared to over all 
PSNR. This result is attractive because discontinuities at block boundaries which are 
the most visually annoying effect, are reduced. Hence, this computationally efficient 
scheme reduces blocking artifacts considerably with slight improvement in PSNR. 

1.7.6 Reduction of Block-boundary Discontinuities: Second Ap- 
proach 

A second approach for discontinuity reduction is proposed. In this algorithm, the 
discontinuities are captured in the same way but it uses the SVD based filtering scheme 
for their reduction. It reduces the blocking artifacts considerably but its computational 
complexity is slightly more because SVD algorithm has to be applied to each of the 
block. The PSNR achieved by this algorithm is also slightly less as compared to the first 
approach. 

1.7.7 Blocking artifact reduction by eliminating some DCT 
coefficients of concatenated block of two adjacent blocks 

This algorithm reduces the blocking artifacts by eliminating some selected DCT coeffi- 
cients of concatenated block of two adjacent blocks. This DCT domain algorithm has low 
computational complexity, as no filtering is required. The availability of fast algorithm 
for DCT computation makes this algorithm suitable for low bit rate coding applications. 
The selection of first row and first column of the DCT of concatenated block further 
reduces the computation complexity. This computationally efficient algorithm also gives 
very good result in terms of PSNR, discontinuity measure and visual quality. 

The performance analysis and simulation results show that the proposed techniques 
are very competitive compared to previously published works. 
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Motivations 

Therefore the motivations behind the work presented in this thesis are: 

• Uncompressed video and audio data are huge and there is a big problem for their 
storage and network communication. Therefore, data compression is required. As 
the compression ratio of lossless methods (e.g., Huffman, Arithmetic) is not high 
enough for image and video compression, a lossy compression scheme that can 
provide higher compression ratio is needed. 

• Very good energy compaction and decorrelation properties of DCT motivated to 
work on the DCT based image compression scheme. The performance of DCT 
based compression scheme is limited by blocking artifacts. So, the main objective 
was to find a DCT based scheme that can minimize these blocking artifacts. 

• Most of the existing popular standards for image compression also use DCT based 
image compression schemes. This was also one of the main reasons for taking up 
this problem. Although at the beginning a number of possibilities for modification 
of coding schemes are studied but looking at these existing standards the work is 
centered around the postprocessing of DCT coded images. 

• DCT-based image compression is very popular and finds various applications. 
Improvement in quantization, encoding and postprocessing can increase the ca- 
pabilities of DCT-based image compression scheme over other schemes in many 
applications. 

• Low bit rate coding applications motivate to find computationally efficient algo- 
rithms for postprocessing of such images. 

Organization of Thesis 

Chapter 1 introduces various aspects of data compression with an overview of DCT 
based image compression scheme. Various methods of postprocessing of DCT coded 
images are discussed and a state-of-the-art survey on the subject is presented. 

In Chapter 2, a hybrid scheme is presented which takes advantages of adaptive 
filtering in edge area and block boundary discontinuity algorithm in monotone area. 
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Chapter 3 presents an approach based on space- variant/ adaptive filtering. In this 
approach, a 2D HR multiple-notch filter is designed from ID multiple-notch filter. 
This filter is applied to reduce the blocking artifacts from DCT coded images either 
by improving the classification scheme or by directly filtering the blocking artifacts in 
monotone areas. 

In Chapter 4^ the design approach presented in Chapter 3 of 2D multiple-notch filter 
is extended to FIR type filter. This filter is also applied to reduce the blocking artifacts 
from DCT coded images as in Chapter 3. 

Chapter 5 is devoted to some new computationally efficient algorithms. Four algo- 
rithms to reduce the artifacts in DCT domain are presented. 

Chapter 6 concludes the major contributions of this thesis and suggests the scope for 
future research work in this area. 



Chapter 2 


A Hybrid Scheme for the 
Postprocessing of DCT Coded 
Images 

2.1 Introduction 

In block discrete cosine transform (BDCT) based image compression schemes the re- 
constructed image from highly compressed data has noticeable degradation due to the 
blocking artifacts. In postprocessing of DCT coded image, filtering of image without 
considering the local image statistics often causes the loss of high frequency detail such 
as edges. Therefore, a space- variant or adaptive filtering techniques are used to overcome 
this problem. Both adaptive and space-variant filtering requires a classification scheme 
to categorize pixel into different class for adaptation. Then different spatial filters are 
used to remove coding artifacts according to class information. In these schemes, a lower 
order filter is preferred to save the processing time. However, to achieve better spatial 
adaptiveness, short-tap filters are preferred in smooth area near to edge but long-tap 
filters are required in larger smooth region. Thus, in these schemes the processing time 
for monotone block is more than that of edge block. Thus a better scheme that can 
reduce the over all processing time is required. In this Chapter, a hybrid scheme is 
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proposed^ in which the DCT coefficients of monotone block axe compensated to achieve 
minimum block-boundary discontinuity whereas in edge blocks a signal adaptive filter 
is applied to perform the filtering along the edge. This scheme is named as hybrid 
scheme as the processing is done in both DCT as well as spatial domain by two different 
algorithms. The compensating DCT coefficients for monotone blocks are calculated 
from the DCT of the block-boundary pixel differences. In this calculation, those block- 
boundary pixel differences are made zero whose the neighboring block is an edge block. 
In proposed scheme, the edge information is extracted and used for the classification 
of image blocks. The signal adaptive filter also uses edge-map to perform the filtering 
approximately parallel to edge to avoid blurring. Thus, the blocking artifacts are reduced 
more efficiently by this scheme. 

As discussed in Chapter 1, in BDCT based image compression, high compression ratio 
are usually achieved by discarding information about the BDCT coefficients that are 
considered unimportant and yields image that exhibit the visually annoying blocking ar- 
tifact. It is done by the quantization of BDCT coefficients. This process is an irreversible 
one that results in degradation of coded image. The blocking artifact manifest itself as 
an artificial boundary among the pixels of adjacent blocks and constitutes a serious 
bottleneck for many important visual communication applications where the pleasing 
image is required at very high compression ratios. Blocking artifacts are the major 
degradation in block-based image coding techniques, especially at higher compression 
ratio. Thus, the objective of postprocessing is to maximally remove noise introduced 
by quantization process without sacrificing the image sharpness. Low pass filtering is 
inadequate to remove various types of noise present in different area of blocky image. 
Low pass filtering reduces grid noise in monotone area but it doesn’t reduce the staircase 
noise. So, nonlinear space-variant or adaptive filter is required to reduce various types of 
noise. Several postprocessing algorithms have been proposed to reduce blocking artifacts 
of block coding system [1,3-5,22,24,27,56]. In non-linear space-variant filtering [1], the 

image is divided into monotone area and edge area using edge information obtained 
^Part of this Chapter represents work accepted for presentation and publication, see [54, 83] 
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from detecting edge by gradient operator. Here two-dimensional (2D) lowpass filter is 
used to reduce grid noise in monotone area, whereas a directional one-dimensional (ID) 
filter is used to reduce staircase noise along the edges. In adaptive filtering [3,24,27,56], 
the coefficients of 2D filter are chosen by edge information of the image. No filtering 
is done at edge pixels. Weighted or un- weighted filtering is performed at the pixels 
in the monotone area or near the edge. In [56], 2D multiple-notch filter is used to 
reduce the blocking artifacts in monotone area. Both nonlinear space-variant filtering 
and adaptive filtering require classification of image blocks, which is based on available 
edge information extracted from the received blocky image. Hence the performance of 
space- variant or adaptive filtering scheme degrades. In references [2,5], the blocking 
artifacts are reduced by processing the image in DOT domain itself. 


2.2 Proposed Scheme 

In postprocessing of DCT coded image, a good classification scheme is required because 
single algorithm cannot give the best performance in different types of regions. To 
maintain the sharpness of the decoded image filter coefficients must be selected carefully 
as the filtering without considering the local image statistics often cause the loss of high 
frequency detail such as edges. Hence, space-invariant lowpass filter is not suitable as 
it will blur the edge and degrade the image visual quality. Space-variant or adaptive 
filtering technique can be used to overcome this problem. An adaptive filtering tech- 
nique uses classification and edge detection to categorize pixel into different class for 
adaptation. Then different spatial filters are used to remove coding artifacts according 
to class information. Classification of image segments is essential to both space-variant 
and adaptive filtering which attempts to exploit local statistics of image regions and 
the sensitivity of human eyes. As each block of image has been properly classified, 
one or more appropriate filters can be used to smooth out the coding artifacts. To 
avoid blurring, LPF is applied to areas that are not masked by the presence of detail 
information such as smooth region. A 2D separable lowpass FIR filter provides one 
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popular choice to artifact reduction in smooth regions. As far as possible, lower order 
filter is taken to save the processing time. To achieve better spatial adaptiveness, the 
length of the LPF may vary. It is also found that the human visual system (HVS) is 
more sensitive to blocking artifacts in monotone regions as compared to edge regions. 
Therefore, a strong long-tap smoothing filter is required in monotone area. For smooth 
area near to edge, short-tap filters such as filter of size 3x3 are preferred. 

Based on above discussion, a hybrid scheme for the postprocessing is being proposed 
in this Chapter. In the proposed scheme, the edge information is extracted and used for 
the classification of image blocks. For edge blocks, a signal adaptive filter performs 
the filtering along the edge to reduce the staircase noise. This filter also uses the 
same edge information (edge-map). In the monotone area, the DCT coefficients are 
compensated for loss of their accuracy using minimum block-boundary discontinuity 
criterion in transform domain itself. These two algorithms are discussed in the following 
two subsections. 

2.2.1 DCT domain Block-boundary Discontinuity Minimiza- 
tion Algorithm for Monotone Area 

In transform coding, the DCT coefficient is quantized with a quantization step size that is 
generally determined by the location of coefficient and color component. If no other noise 
exists, then quantization error for the coefficient is half of the corresponding quantization 
step size. Since the quantization processes is in transform domain, the quantization noise 
affects the pixel values over all the spatial locations. So, it is better to reduce it in DCT 
domain itself. Blocking artifacts are the result of quantization process required for image 
compression. These blocking artifacts are visible as an axtificial boundary among the 
pixels of adjacent blocks. Thus, minimizing the discontinuity of pixel values over block 
boundaries can reduce these artifacts. This algorithm is applied in monotone area since 
blocking artifacts are more visible in these areas and does not contain high frequency 
details. 
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Figure 2.1: A block and its adjacent top, bottom, left and right blocks to calculate the 
block-boundary discontinuity. 

In proposed scheme the minimum discontinuity of pixel values over block boundaries 
is achieved by compensating for the loss of accuracy of transform coefficients by the 
algorithm suggested by Jeon and Jeong in [5] with some modification. In their algorithm 
they have defined block-boundary discontinuity measure Dk{f) for block as the sum 
of squared pixel differences over the four block-boundaries to quantify the blocking 
artifacts. 

Dt(f) = T,^^^\h-F(N -l,y)- h(0,y)f + T,y^'[fM(%y) - MN - I,;/)]" 

+ Ej'=o‘[A-i(^.«'-i)-A(x,0)P + E"y[A+i(^,o)-A(i,JV-i)P ^ ■ 

where fk-p, fk+N, fk-i f-nd fy+i refer to pixels in the top, bottom, left an right 
blocks of the block, respectively. The total block discontinuity is given by: 

D(f) = E Di,(f) (2.2) 

fc=l 

In this scheme compensation signal Af{x,y) is added to the dequantized image 
fo get resultant signal / {x,y) as: 


/ (x, y) = fqix, y) + A/(x, y) 


(2.3) 
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The compensating signal is estimated for each block to obtain the minimum block- 
boundary discontinuity. This is a constraint least square problem where the mini- 
mization is performed in DCT domain. In [5] the compensating DCT coefficients are 
calculated as: 

0-00 = \/A[Gfc_p(0) -|- Gk+p{0) 4- Gfc_i(0) + Gifc+i(0)]/4 

Ooi = + Gfc 4 ,p(l) -I- Gfc_i(0) — Gfc4.i(0)]/[2 ■+- 4cos^(7r/2A’)] 

Oio = '^/A[Gfc_p(0) — Gfc-|.p(0) -t- Gfc_i(l) 4- Gfc+i(l)]/[2 4- 4cos^(7r/2A)] 

On = \/iV[Gfc-p(l) — Gji; 4 .p(l) 4- Gfc_i(l) — Gfc+i(l)]/4\/2cos(7r/2A')] 

where Gk(r) is the one dimensional DCT coefficient of pixel difference over block- 
boundary gf;±p and ^^±1 which are estimated simply by difference of dequantized values 
to further simplify the processing. To control degree of compensation weight factor A is 
multiplied to the estimated compensating coefficients. In this case a weight factor near 
0.5 is found to be appropriate. 

In addition to applying this algorithm only in monotone area, some modifications 
have been made. One main difference between the proposed algorithm and the algorithm 
proposed in [5] is in the calculation of block-boundary differences. In this calculation, 
those block-boundary pixel differences are made zero whose the neighboring block is 
an edge block. This is based on the fact that characteristics of edge area is quite 
different from monotone area as we can observe that the artificial discontinuities at 
block-boundaries due to blocking artifacts are more visible in monotone or slowly varying 
areas. This modification improves the performance of the algorithm. 

2.2.2 Spatial Domain Signal Adaptive Filtering for Edge Area 

As discussed in [1], a directional filter is required to reduce staircase noise along the 
edges. Therefore, in proposed hybrid scheme a directional or signal adaptive filter is 
applied in edge area. This adaptive filtering scheme uses the edge-map obtained from 
an edge detection algorithm (Sobels, Canny operator etc.). Depending on this edge- 
map, some of the filter coefficient are made zero. In proposed signal adaptive filtering 
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Figure 2.2: Filtering mask of the signal adaptive filter. 

(SAF), if the center E of filtering mask (see Fig 2.2) is at edge pixel of the block then 
all surrounding coefficients A, B, C, D, F, G, H, and I are made zero and thus the value 
of edge pixel remains unchanged. If the center of mask is at non-edge pixel in the block, 
then those surrounding values that are at edge pixels are decreased to zero whereas value 
at the center of mask is increased by the same amount. 

2.2.3 Comer Outliers Detection and Replacement 

As discussed in Chapter 1, corner outliers are visible at the cross point of M x M block 
when the comer pixel of one block is very large or very small as compared to comer 
pixels of the neighboring blocks. This condition is tested to detect the comer outliers 
by comparing each corner pixel with its neighbors. If the corner outliers exist, then the 
former and its neighbors are replaced by its weighted average to reduce this effect. The 
replacement scheme for this purpose is discussed in [3] is given below: 

A' = {4A + B + C + 2D + 4)/8 

A[ = (A' +3Ai + 2)/4 (2.5) 

Aj = (A + 3 A 2 + 2) /4 

As shown in Fig. 2.3, the pixel A is detected as comer outliers and the pixels A, Ai and 
A 2 is replaced by A', A^ and Aj. 
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igure 2.3: (a) Presence of comer outliers, (b) Pixels used in weighted averaging for the 
iduction of corner outliers. 


1.3 Results 

1 this section, results of proposed algorithm applied on 256 x 256 Lenna image has been 
resented. The algorithm is implemented with MATLAB. Various results are given in 
'able 2.1 in which the results of proposed algorithm are compared with baseline JPEG 
ke DCT decoded image. Peak signal to noise ratio (PSNR) is defined as: 

2552 

PSNRiin dB) = 10 logioij^) (2-6) 

Where 255 is the peak signal for 8 bit PCM and MSE is mean square error given by 

= Ac E (/»."- A»)'l (2-’') 

i=0 j=0 

Vhere fmn and are the pixel values at position (m, n) of original and decoded 
mage respectively. As the PSNR is only the rough indicator of image quality and does 
lot reflects the blocking artifacts, two new discontinuity measures are defined. The 
dock-boundary PSNR (BPSNR) is defined in the same way as PSNR but only one pixel 
rom both side of block boundary are considered for the calculation of mean square 
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Table 2.1; Comparison of the performance of hybrid scheme on Lenna image. 


Algorithms 

PSNR in dB at 
quality factor q 

Boundary 

PSNR in dB at 
quality factor q 

Boundary Disconti- 
nuity in (10® j at 
quality factor q 

q=5 

q=7 

q=10 

q=5 

q=7 

q=10 

q=5 

q=7 

q=10 

Original 

- 

- 


- 

- 

- 

4.28 

4.28 

4.28 

DOT coded 

25.32 

26.75 

28.09 

15.02 

16.51 

17.96 

10.9 

9.22 

7.93 

Signal 

Adaptive 

filtering 

25.17 

26.12 

27.11 

15.15 

16.16 

17.22 

8.89 

7.85 

7.07 

block-boundary 

discontinuity 

minimization 

25.60 

27.00 

28.40 

15.52 

16.88 

18.31 

8.92 

7.67 

6.65 

Proposed 

algorithm 

25.72 

26.95 

28.31 

15.62 

16.84 

18.22 

8.66 

7.62 

6.68 


error (MSE) whereas the block-boundary discontinuity measure is defined as the sum of 
square of difference of pixel values across the four block-boundaries. 

The performance of the proposed algorithm improves as compression ratio increases. 
The reduction of discontinuity signifies (see Table 2.1) blocking artifacts are reduced 
which can also be observed visually in Fig. 2.4, Fig. 2.5 and Fig. 2.6. These figures and 
Table 2.1 also shows that neither of the two algorithms used in hybrid scheme are able 
to give good results if they are used to process whole image. It can also be observed 
from Table 2.1 that the improvement in PSNR for block-boundary pixels is more as 
compared to over all PSNR. 
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(c) (d) 

figure 2.4: Reconstructed Lenna image (compressed at Quality Factor 5) by various 
ilgorithm (a) Simple DOT decoded, (b) Postprocessed by signal adaptive filtering, (c) 
^ostprocessed by block-boundary discontinuity minimization algorithm, (d) Postpro- 
:essed by proposed hybrid scheme. 
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(c) (d) 

Figure 2.5: Reconstructed Lenna image (compressed at Quality Factor 7) by various 
algorithm (a) Simple DOT decoded, (b) Postprocessed by signal adaptive filtering, (c) 
Postprocessed by block-boundary discontinuity minimization algorithm, (d) Postpro- 
cessed by proposed hybrid scheme. 
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(c) (d) 

Figure 2.6: Reconstructed Lenna image (compressed at Quality Factor 10) by various 
algorithm (a) Simple DOT decoded, (b) Postprocessed by signal adaptive filtering, (c) 
Pcotprocessed by block-boundary discontinuity minimization algorithm, (d) Postpro- 
cessed by proposed hybrid scheme. 


Chap. 2 A Hybrid Scheme for the Postprocessing... 


51 


2.4 Conclusions 

In this Chapter a hybrid scheme for postprocessing of DOT coded image is proposed 
which takes the advantage of adaptive filtering in edge area and block-boundary dis- 
continuity minimization algorithm in monotone area. Thus, the processing of image 
is done in both spatial and transform domain. In this algorithm, in order to reduce 
the processing time, a lower order signal adaptive filter is applied in edge area whereas 
block-boundary discontinuity minimization algorithm is used in monotone area. Block- 
boundary discontinuity minimization algorithm reduces the blocking artifacts with low 
computational cost. Thus, proposed algorithm reduces the blocking artifacts consider- 
ably without introducing much blurring in the edge area. The results are verified by 
comparing boundary PSNR and block-boundary discontinuity (see Table 2.1). This 
can also be observed from the results presented in Fig. 2.4, Fig. 2.5 and Fig. 2.6. 
These results also show that neither of the two algorithm used in hybrid scheme can 
give good results if they are applied to process complete image. It is found that 
improvement in PSNR for block-boundary pixels is more as compared to over all PSNR. 
This result is attractive because discontinuities at block-boundaries, which are the most 
visually annoying effect, are reduced. Thus, this hybrid scheme reduces blocking artifacts 
considerably without introducing much blurring in the edge areas. This scheme also 
provides slight improvement in PSNR. 


. IV 



Chapter 3 


Design and applications of 2D HR 
multiple- notch filter 


3.1 Introduction 

* 

The reconstructed image from highly compressed JPEG data has noticeable degradation 
due to blocking artifacts. Linear, space-invariant filtering is inadequate to remove these 
artifacts. So, space-variant or adaptive filtering is required. These schemes are based on 
edge information extracted from the received blocky image, which contain false edges 
due to blocking artifacts. Based on analysis in frequency domain, it has been found that 
artifacts in decoded image have periodic structure. These periodic-textures become more 
obvious after gradient operation since discontinijities due to block coding are highlighted 
in gradient image. According to signal processing theory, periodic texture will result in 
some peaks in frequency domain. Here, peaks are due to false edges in gradient image 
or due to blocking discontinuities in decoded image. A two-dimensional (2D) multiple- 
notch filter can kill these peaks to reduce the artificial edges due to blocking artifacts 
in gradient image. In this Chapter^, a 2D HR multiple-notch filter is designed from 
one-dimensional (ID) multiple-tone filter using simple algebraic method. This approach 
is an extension of method used by Pei e^ al. [6] in the domain of 2D. Since discontinuities 
^Part of this Chapter represents work published before, see [56] 
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due to blocking artifacts are more visible in monotone area, this filter can be applied 
directly in these areas to remove them. Thus, improvement in the performance of space- 
variant/adaptive filter by removing of false edges from edge of blocky decoded image 
and reduction of blocking artifacts directly in monotone area, are the new possible 
applications of multiple-notch filter which are considered in this Chapter. The filtering 
is performed in DCT domain to reduce the computational complexity. 

3.2 Objective 

In BDCT based image compression, as discussed in Chapter 1, the blocking artifacts are 

main cause of degradation, especially at higher compression ratio. Low pass filtering is 

inadequate to remove various types of noise present in different area of blocky image. 

Low pass filtering reduces grid noise in monotone area but it does not reduce the staircase 

% 

noise. So, non-linear space-variant or adaptive filter is required to reduce various types 
of noise. These schemes require classification of image, which is based on available 
edge information extracted from the received blocky image. This edge information also 
contains false edges due to blocking artifacts. Hence the performance of space-variant 
or adaptive filtering scheme degrades. 

Several postprocessing algorithms [1,3-5,22,24,27,56] have been proposed to reduce 
blocking effect of block coding system which are discussed in brief in Chapter 1. In non- 
linear space- variant filtering [1], first the image rs divided into monotone area and edge 
area using edge information obtained from gradient thresholding and then a suitable filter 
is applied in each of these two areas. In adaptive filtering [3,24,27,56], the coefficients of 
2D filter are chosen by edge information of the image. Weighted or unweighted filtering 
is performed at the pixels in the monotone area or near the edge but the edge pixel 
remains unaffected. 

Based on analysis in frequency domain [8], it has been found that blocking artifacts 
in decoded image appears periodic in the structure. Artifacts are expected to have 
periodic texture because block size is fibced in block coding. This kind of texture hides 
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hin the image and it is difiicult to detect them. However, this periodic-texture 
omes more obvious after gradient operation since discontinuities due to block coding 
ighlighted in gradient image. According to signal processing theory, periodic texture 
I result in some peaks in frequency domain. Thus, periodic texture can be detected 
re conveniently in frequency domain. In case of the DCT coded image, the artificial 
es in the gradient of the decompressed image are due to the blocking artifacts. These 
■ficial edges result in multiple peaks in DCT domain. Thus, a 2D-multiple notch filter 
equired to kill these peaks to reduce the artificial edges in the gradient image. 

In this Chapter, the design of 2D multiple-notch filter is considered and the possibility 
ising it for the reduction of blocking artifact, which is the main objective of this work, 
been discussed. Proposed scheme for blocking artifact reduction is based on the 
ce- variant/ adaptive filtering but the use of 2D HR multiple-notch filter is new. This 
T is applied for artificial edge removal to improve the classification scheme of space- 
ant/ adaptive filtering. In addition to this, it can also be applied in removing the 
:king artifacts in monotone area directly as artificial discontinuities due to blocking 
facts are more visible in these areas. The design of 2D HR multiple-notch filter will 
discussed in section 3.5 whereas its applications will be discussed in section 3.7. In 
ion 3.4 the review of the design of ID multiple-notch filter has been presented as the 
gn of 2D IIR multiple-notch filter is based on this. In the following section 3.3, the 
ring in DCT domain is briefly discussed. 

} Filtering in DCT domain 

i problem of 2D filtering in DCT domain is an important problem in the area of 
jessing and manipulating of images and video streams compressed by DCT-based 
hod. Convolution multiplication property (CMP) for DCT is given in [57-62]. Like 
solution multiplication property for discrete Fourier transform (DFT) converts the 
ring operation to simple element-by-element multiplication, CMP property of DCT 
also be used to get similar results. First Chen and Fralick [57] gave the CMP for 
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'CT which was used for lowpass filtering by Ngan and Clarke in [58]. Then, Chitprasert 
id Rao [59] derived a much simpler CMP. They proposed two approaches. In first 
iproach, a multiplication in spectral domain results in a circular convolution in spatial 
imain plus a constant DC term. Their second approach is even simpler in which the 
lultiplication of the DCT of signal sequence with the DFT of filter sequence results 
1 circular convolution of the folded signal sequence and the filter sequence. Most of 
le works on CMP do not implement linear convolution, but rather a convolution with 
folded input sequence, called symmetric convolution. The convolution-multiplication 
roperty can be used in filtering the images. To filter 256x256 pixels of test image, a 
lock size of 8 x 8 may be selected. To perform filtering with a separable 3x3 2D 
Iter, first each of the ID filter is Fourier transformed by 16 point DFT. The resulting 
equency responses are real and even. Then 2D frequency responses are created by 
lultiplication of the ID versions. Finally 8x8 DCT of test images are multiplied by 
le filter matrices. 

Symmetric convolution is the convolution mode of discrete trigonometric transforms 
DTT) [60]. The DCT is one member of DTT family. Symmetric convolution can be 
ficiently computed by taking inverse DTT of element-by-element product of forward 
•TT’s of the inputs. We must use the appropriate type of DTT for each type of 
fmmetric convolution. Symmetric convolution provides the systematic way to convolve 
tunmetric FIR filters with symmetrically extended data. These various possibilities 
f symmetrically extending the two sequences at both ends result in various types of 
yunmetric convolutions. 

1.4 Review of design of ID HR Multiple-notch Filter 

'he design of ID multiple-notch filter is available in [7]^ [49]. These techniques are 
Iso discussed in brief in Chapter 1. In these approaches ID multiple-notch filter is 
esigned from the allpass filter. The specifications of notch filter is first transformed 
ito that of all-pass filter and effective approach to design this desired all-pass filter is 
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developed. All-pass filter A(z) is designed such that the phase response A(a;) satisfies 
2M requirements, then (l-|-A(z))/2 are the desired notch filter. Here 2M is the order of 
the filter. The design method proposed by Joshi and Dutta Roy [49] is similar to the 
method proposed by Pei at el. [7]. In [49], ID multiple-notch filter is realized by the 
cascade of second order allpass filter. Now, the design approach proposed by Pei in at 
el. [7] will be discussed in detail. The input of notch filter has the following form; 


M 

x{n) = s{n) + '^Ak sm{nujivk + (pk) 

k=l 

= s(n) + cl(n) 


(3.1) 

(3.2) 


where M is number of sinusoidal interferences, s(n) is the desired signal and d(n) is 
sum of all sinusoidal interferences, ui^k is frequency for k=l, 2, ..., M. To remove these 
sinusoidal interferences, the specifications of ideal notch filter is given by 


H{eP'^) 


0 for ui = UMk, k = 1,2,3, ...,M 

1 otherwise 


(3.3) 


Specification Tramsformation 


The specifications of notch filter are first transformed into that of all-pass filter using 
the relation H{z) = (1 -I- A{z))/2. As a result, the notch filteY design problem reduces 
to an allpass filter design problem. The transfer function of 2M order allpass filter A(z) 
is given by 


I I j-w ^ — SAif-f-l j — *2i]^ 

0'2Mi + + CLli^i + 

1 + ttiiZi ^ -1- .... -t- a2Mi^i 


(3.4) 


As magnitude response of A(z) is equal to unity for all frequency, its frequency response 
can be written as 


A{e^^) = 


(3.5) 


where 0a{<^) is the phase response. Thus the frequency response of notch filter can be 
written as 


fr(ej‘^) = i(l -h 

ji 


(3.6) 
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he 9a{i^) of a stable allpass filter is 0 when w = 0, —2M'K when oj = tt, and is required 
) decrease monotonically with increasing frequency. Thus, when w goes from 0 to tt 
idians, the phase goes from 0 to —2M'k. Therefore 

1. There exits M frequency points wi < a;2 < ... < ojm such that 0.4 (a;„) = — (2n— l)7r, 

or = 0 for n=l, 2, ...M. 

2. There exits M frequency points < ... < such that = 

-(2n - l)7r + f , or I |=1 |(1 - j) |= ^ for n=l, 2, ...,M. 

3. There exits M frequency points < ... < such that 9a{<^^^) = 

-(2n - l)7r - f , or 1 |=| |(H-i) |= ^ for n=l, 2, ...,M. 

4. There exits M+1 frequency points 0 = < ... < = tt such that 

0.4(0;®) = — (2n)7r, or = 1 for n=l, 2, ...,M. 

rhen H(z) is forth-order notch filter, i.e. for M=2, following assignments of the phase 
.4(60) are needed 
) 9A{oJNn) = -{2n - l)7r 
) 9A{oJNn - BWn/2) = -{2n - l)7r + 7r/2 
) 9A{oJNn + BWJ2) = -{2n- l)7r - 7r/2 

;here n=l, 2,...M and notch frequency points o;jv„ satisfies cUr^i < un 2 < ■■■ < 
loreover, if BWn is very small, 0.4(a;iVn) = .~(2w — l)7r and 9A{u}Nn — BWn/2) = 
-(2ti — l)7r + 7r/2, then to a first order approximation it can be shown that 

0A{(^Nn + BWJ2) -(2n - l)7r - 7r/2 (3.7) 

."'hus for small rejection bandwidth BW^, the above three assignment reduces to the 
ollowing two assignments 

1) 9A{<^Nn) — — (2n. — l)7r 

2) eAiuJNn - BWJ2) = -{2n - l)ir + ir/2 
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ter suitable arrangement, these two assignments are equivalent to the following con- 
tion in 2M frequency-sampling points. If the frequency points are 

1 -BWfiiil 

= (3.8) 

en the desired phase response is specified by 

= -(2[^l - l)jr + 1(1 - (3,9) 

lere i=l,2,...2M and [x] denotes the largest integer which is smaller than or equal to 
and mod{x, 2) denotes the remainder when x is divided by 2. Thus the specifications 
notch filter have been transformed into specifications of allpass filter. 


3sign Procedure 


le phase response of of A(z) can be written as 


= —2Muj -f 2 arctan( 


sin (feu;) 

1 + 2Mak cos(A:a;) 


(3.10) 


om equation (3.9), the phase response 6a{<jj) given at 2M points Wj, i=l,2,...2M, we 
tain a set of equations as 


Efc=i 2Mak sm{koJi) 

1 + Efc=i 2Majfc cos{koji) 


= tan(^i) i = l,2. 


2M 


(3.11) 


lere = |[0^(a;t) 2Ma;i]. After some manipulations, this expression can be written 


2M 

'^[sm{ku}i) — tan(/3t) cos{ku}i)]ak = i = 1, 2, ..., 2M (3-12) 

k—l 

ich is a set of linear equation in the filter coefficients afc. The set of linear equ^ition 
matrix form can be given by 

Qa = p (3.13) 

ere 


[ai,a2, —02 m]^ 


p = [tan^ijtan^, ...tanP2MT 
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and the elements of matrix Q are given by: 

Qik = sin{kLOi) - tan^iCos{kUi) i = 1, 2, 2M fc = 1, 2, 2M (3-14) 

By solving these linear simultaneous equations, the desired solution is given as 

a=Q-^p (3.15) 

Thus the realization of notch filter H(z) is equivalent to realization of allpass filter A(z). 

H{z) = [1 + A(z))\li (3.16) 

Due to mirror image S 5 Tiimetry relation between the numerator and denominator poly- 
nomials of allpass filter, the notch filter can be realized by computationally efficient 
lattice structure. 


3.5 Design of 2D HR Multiple-notch Filter 

In proposed approach, the 2D HR multiple-notch digital filter is designed from one- 
dimensional (ID) multiple-tone (narrow bandpass) filter. The ID multiple-tone filter is 
designed from allpass filter [7] in the similar way as discussed in section 3.4- The design 
approach of 2D HR multiple-notch filter is based on the concept used in [6] for the 
design of 2D HR single-notch filter. This approach is discussed in Chapter 1 where the 
2D single-notch filter is designed by cascading 2D-parallel line filter with 2D straight-line 
filter. In proposed approach, a special 2D multiple-notch filter is obtained by cascading 
2D multiple vertical-line filter with 2D multiple horizontal-line filter. The block diagram 
of 2D HR multiple-notch filter of proposed method is shown in Fig. 3.1. From the block 
diagram it is clear that the design of two multiple-parallel line filters are required to 
design a 2D multiple-notch filter. The specifications of 2D multiple-notch filter is given 

by: 




0, for {Wi,UJ 2 ) = { 0 JlNk^ 0 J 2 Nk) 

1, otherwise 


(3.17) 
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Figure 3.1: Block diagram of a 2D HR multiple-notch filter. 


where k=l,2, M; for M notch frequencies. Now to design such filter, simple algebraic 

method is used. In this method we start with writing the specifications of corresponding 
2D multiple-parallel line filters as: 




1, for OJi = ±L0iNk 
0, otherwise 


(3.18) 


where i=l,2 corresponds to vertically parallel multiple-line filter and horizontally parallel 
multiple-line filter, respectively. These kind of filters can be designed by choosing 


H{zi,Z 2 ) as: 


Hi{zi,Z2) = Hmtf{z)\z=zi for i = l,2 


(3.19) 


where Hmtf{z) is a ID multiple-tone filter and i = 1,2 corresponds to vertically and 
horizontally parallel multiple-line filter, respectively. Thus multiple parallel line filters 
can be obtained using ID multiple-tone filter, which can be designed from allpass filter. 
As discussed in section 3.4, if the allpass filter A{z) is designed such that the phase 
response A(a;) satisfies 2M requirements, then (1 — A{z))/2 are the desired multiple- 
tone filter. Here 2M is the order of the filter. Thus 


Hmtf = [l-Aiz)]/2 


(3.20) 


where A(z) is allpass filter whose transfer function is given by : 


A{zi) 


JL _L_ n 1 -—2M 

0'2Mi + + 

1 + dii^i ^ 


(3.21) 





PHASE RESPONSE x jt AMPLITUDE 
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Figure 3.2; Frequency response of (a) ID HR allpass filter and (b) ID HR multiple-tone 
filter. 
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/Vhere i = 1, 2 and 2M is the order of the eaxdi allpass filter to get M notch frequencies. 
The coefficients of allpass filter A{z) can be obtained using method proposed in [7]. 

Thus realization of proposed notch filter is based on the realization of an allpass 
liter. The desired 2D multiple-notch filter can be obtained from these properly designed 
vertical parallel line filter Hi{zi,Z 2 ) and multiple horizontal line filter H 2 {zx,Z 2 ) ■ The 
ransfer function of 2D multiple-notch filter is given by; 

Hn{zx,Z 2 ) = 1 - Hx{zx, Z 2 )H 2 {Zx,Z 2 ) (3.22) 

Che frequency response of the ID multiple-tone filter and corresponding allpass filter 
xe shown in Fig. 3.2. The block diagram of 2D multiple-notch filter is shown in Fig. 3.1 
srhereas its amplitude response is shown in Fig. 3.3 (a). 


5.6 Design Example 


n this section, a design example has been presented which was performed with MAT- 
AB. 


p 1: The notch frequencies for a 2D multiple-notch filter are calculated from the block 
size. The notch frequencies and corresponding bandwidths of this filter are selected 
for the block size of 8 x 8 as 


wi = 0 


BWi = 0.027r 


' UJ 2 = 0.257r 


BW2 = O.OItt 


cus = O.SOtt 


BWa = O.OItt 


a ;4 = 0.757r RW 4 = O.OItt 

Ws = TT BW5 = 0.02tt 

In this case, these frequencies and bandwidths are used in the design of two 
identical ID multiple-tone filter. 

p 2: Using method proposed in [7], we obtain the filter coefficients of eighth-order all- 
pass filter as: 
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Cl = 1.9201 
as = 1.8869 
as = 1.8300 
ar = 1.7969 
ag = 1.7969 


ag = 1.8806 
a4 = 1.8622 
ae = 1.8437 
as = 1.8622 
aio — 1.8622 


3p 3: ID multiple-tone filter coefficients are obtained from equation (3.20). The fre- 
quency response of this filter along with the corresponding allpass filter is shown 
in Fig. 3.2. 

3 p 4; Using proposed method, we obtain the transfer function of 2D multiple-notch filter 
from Hmtf(z). The amplitude response of this filter is shown in Fig. 3.3 (a). 

ep 5 ; DCT coefficient of block size M x M is multiplied by filter matrix of equal size. 


3.7 Applications of 2D multiple- notch filter 


This filter can be used for the removal of single or multiple sinusoidal interferences. As 
discussed earlier, the discontinuities at block boundaries of decoded image are due to 
blocking artifacts, which have the periodic structure and results in some peaks in DCT 
domain. A 2D multiple-notch filter can be used to kill these peaks to reduce the blocking 
artifacts firom DCT coded images in the following two ways. 

3.7.1 Improvement in the performance of space- variant/ adap- 
tive filter by removal of false edges from edge-map of 
blocky image 

[t is obvious from the property of DCT and it can also be observed that the DCT of 
square grid of line spacing of eight in both horizontal and vertical direction has peaks at 
a interval of 64 in both directions. This grid can be removed by killing these peaks by 
suitable notch filter. Here, this fact is verified as shown in Fig. 3.3 (b) and Fig. 3.3 (c). 
The notch frequencies for this multiple-notch filter are calculated from block size. It has 
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also been observed that as the grid of lines due to blocking artifact in the edge-image is 
not complete, these peaks remain at the same position but their spread increases. So, 
at high compression ratio, when grid in the edge-map of DCT coded image is visible, 
this grid or false edges can be removed using proposed 2D multiple-notch filter. This 
fact is tested for DCT coded 256 x 256 Lenna image and found to be correct as shown 
in Fig. 3.4 and Fig. 3.5. 

3.7.2 Reduction of blocking artifacts using 2D multiple-notch 
filter 

At high compression ratio, the discontinuities due to blocking artifacts are more pro- 
nounced in monotone area so this filter can be applied directly in these areas. In 
proposed space- variant/adaptive filtering approach, 2D multiple-notch filter and signal 
adaptive filter is applied in the monotone area and edge area, respectively. In space- 
variant/adaptive filtering, the image is first divided into monotone area and edge area 
then in monotone area lowpass fiilter is applied whereas in edge area directional or signal 
adaptive filter is applied. A 2D multiple-notch filter can be applied in monotone area 
because the periodic texture due to blocking artifacts are more visible in monotone areas 
as compared to edge areas. Moreover, the actual edges are blurred if the same filter is 
applied in edge areas. Therefore, in this algorithm a signal adaptive filter discussed in 
Chapter 2 is used in edge area. 

Another advantage of this scheme is low computational complexity as filtering is 
performed in DCT domain. The results are shown in Fig. 3.6. These results show that 
blocking artifacts can be reduced by this method. However, the improvement is limited 
to high compression ratios because these periodic textures due to blocking artifacts hides 
in DCT domain at low compression ratios. 
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(e) (f) 

Figure 3.4: Postprocessing of decompressed Lenna image at quality factor= 5. (a) 

Original, (b) Before postprocessing, (c) Edge-map. (d) Edge-map in (c) is filtered by 
2D HR multiple-notch filter, (e) Postprocessed by space-variant algorithm with filtered 
6dge-ni8ip. (f) PostprocBSScd by spsTCc-varisiiit 3<lgorithni without filtored odgc-map. 
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(e) (f) 

Figure 3.5: Postprocessing of decompressed Lenna image at quality factor=7. (a) 
Original, (b) Before postprocessing, (c) Edge-map. (d) Edge-map in (c) is filtered by 
2D HR multiple-notch filter, (e) Postprocessed by space-variant algorithm with filtered 
edge-map. (f) Postprocessed by space-variant algorithm without filtered edge-map. 
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(e) (f) 

Figure 3.6: Postprocessing of Lenna image compressed at quality factor 5 in (a), (b), (c) 
and compressed at quality factor 7 in (d), (e), (f) by various algorithms; where (a) and 
(d) Before postprocessing, (b) and (e) Processed by 2D HR multiple-notch filter with 
Signal Adaptive filter (SAF), and (c) and (f) Processed by Gaussian LPF with SAF 
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Table 3.1: Comparison of the performances of various space- variant algorithms on Lenna 
image. 


Algorithms 

PSNR in dB at 
quality factor q 

Boundary 

PSNR in dB at 
quality factor q 

Boundary Disconti- 
nuity in (l0®j at 
quality factor q 

q=5 

q=7 

q=10 

q=5 

q=7 

q=10 

q=5 

q=7 

II 

o 

Original 

- 

- 


- 


- 

4.28 

4.28 

4.28 

DOT coded 

25.32 

26.75 

28.09 

15.02 

16.51 

17.96 

10.9 

9.22 

7.93 

2D HR multiple 
notch filter 
with SAF 

26.00 

27.41 : 

28.78 

16.37 

17.75 

19.19 

5.66 

4.96 

4.31 

Gaussian 

LPF 

with SAF 

26.11 

27.49 

28.90 

16.60 

17.92 

19.41 

4.97 

4.61 

3.92 
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3.8 Results 

In this section, the results of various algorithms applied on reconstructed blocky 256x256 
Lenna image have been presented. These algorithms are implemented with MATLAB. 
The results of two algorithms are compared with baseline JPEG like DOT decoded image. 
Peak SNR (PSNR) is defined in Chapter 2. As discussed in Chapter 2, the PSNR is only 
the rough indicator of image quality and does not reflects the blocking artifacts, two 

r 

new discontinuity measures are defined. The block boundary PSNR (BPSNR) is defined 
in the same way as PSNR but only one pixel from both side of block boundary are 
considered for the calculation of mean square error (MSE) whereas the block boundary 
discontinuity measure is defined as the sum of square of difierence of pixel values across 
the four block boundaries. Using these measures the performance of various proposed 
algorithms are compared. 

It can be seen from Fig 3.3 that 2D multiple-notch filter can eliminate the square 
grid. This filter can also remove the false edges from the edge-map of blocky lenna image 
as shown in Fig. 3.4 and Fig. 3.5. These figures also show the result of postprocessing of 
DCT coded image by the space-variant filtering algorithm which uses the 2D multiple- 
notch filter for false edge removal in its classification scheme. In the second application 
of 2D multiple-notch filter as discussed in subsection 3. 7.2, the reduction of discontinuity 
signifies that blocking artifacts are reduced which can also be observed visually in 
Fig. 3.6. It can also be seen from Table 3.1 that the improvement in PSNR for block 
boundary pixels is more as compared to over all PSNR. 

3.9 Conclusion 

in this Chapter, a simple algebraic method is used to design 2D HR multiple-notch 
filter. In this approach, at first two ID multiple-tone filters are designed as per the 
specifications of 2D multiple-notch filter and then these two ID multiple-tone filters are 
used to obtain 2D multiple-vertical line and 2D multiple-horizontal line filter. Finally, 
cascade of these two filters, as shown in Fig. 3.1, gives the desired 2D multiple-notch 
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filter. The design of ID multiple-tone filter is based on the design of allpass filter. In 
this Chapter a design example and two new applications of 2D multiple-notch filter 
are discussed. It is found that the reduction of blocking artifacts and removal of false 
edges from edge-map of blocky DCT coded image are the new possible application of 2D 
multiple-notch filter. Furthermore, the space-variant filtering method can be improved 
with a more efficient multiple-notch filter. 



Chapter 4 


Linear-phase 2D FIR Multiple-notch 
Filter for Reducing Blocking 
Artifact from DCT Coded Images 


4.1 Introduction 

The design and applications of 2D HR multiple-notch filter is discussed in Chapter S, 
where it was found that this filter could be used for reducing the blocking artifacts 
from DCT coded images. In this Chapter the design of FIR multiple-notch filter 
and its application in reducing the blocking artifacts from DCT coded images will be 
discussed. As the filter is used for reducing blocking artifacts from DCT coded images, 
it is meaningful to design multiple-notch filter with linear-phase, which provides less 
distortion as compared to the filters with non-linear phase. The Blocking artifacts are 
the major problem with DCT coded image, which is discussed in Chapter 1. Space- 
variant or adaptive filtering is required to reduce these artifacts. These schemes are 
based on edge information extracted from blocky image, which contain false edges due 
to blocking artifacts. Based on analysis in frequency domain, it has been found that 
artifacts in decoded image have periodic structure that corresponds to discontinuiti^ due 
to block coding. These discontinuities are highlighted in gradient image. According to 
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signal processing theory, periodic texture will result in some peaks in frequency domain, 
which are either due to false edges in gradient-image or due to boundary discontinuities 
in decoded image. A two-dimensional (2D) multiple-notch filter can kill these peaks to 
reduce these artifacts. In this Chapter ^ 2D linear-phase FIR multiple-notch filter is 
designed. Discontinuities due to blocking artifacts are more in monotone area so this 
filter can be applied directly in these areas. As discussed in Chapter S, improvement 
in the performance of space- variant/adaptive filter by removal of false edges from edge- 
map of blocky image and the reduction of blocking artifacts directly in monotone area, 
are also the new possible applications of 2D FIR multiple-notch filter. Computational 
complexity is less because the filtering is performed in DCT domain. 


4.2 Objective 

The objective of work presented in this Chapter is once again the reduction of coding 
artifacts. Thus, let us begin with recalling the causes of coding artifacts and their 
reduction methods. The blocking artifacts are the major degradation in block-based 
image coding techniques, especially at higher compression ratio. One main reason of 
blocking artifacts is that blocks are encoded and quantized independently without con- 
sidering the correlation between adjacent blocks. High compression ratios are achieved 
by quantization process in which the information about less important BDCT coefficient 
is discarded. 

As discussed in [1], space-invariant lowpass filter is not suitable because it will blur the 
edge and degrade the image visual quality. It is also found that the human visual system 
(HVS) is more sensitive to blocking artifacts in monotone regions than edge regions. 
Therefore, a strong smoothing filter is required in monotone area. In edge regions the 
smoothing operation tends to introduce undesirable blur, so smoothing of a few pixels 
around block boundaries by directional lowpass filter is sufficient. The information about 
local image characteristics can be extracted from image by edge detection. Thus, one of 


^Paxt of this Chapter represents work communicated for publication, see [63] 
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the best approach of reducing blocking artifact is to classify the image blocks into various 
categories and each is processed by one or two-dimensional filter. Hence, space-variant 
or adaptive filtering depending on local image characteristics is preferable. A number 
of postprocessing algorithms have been proposed in [1-4, 22-29, 56] to reduce blocking 
effect of block coding system as discussed in Chapter 1. In non-linear space-variant 
filtering [1] , the image is classified into monotone area and edge area and then a suitable 
filter is applied in each of the classified area to reduce over all noise introduced in the 
image. In adaptive filtering [3,24,27,56], the coefficients of 2D filter are chosen by edge 
information of the image. 

As discussed earlier, that both non-linear space- variant filtering and adaptive filtering 
require classification of image, which is based on available edge information extracted 
from the received blocky image. This edge information also contains false edges due to 
blocking artifacts. Hence the performance of space-variant or adaptive filtering scheme 
may degrade due to misclassification. Hung at el. [8] proposed a scheme for removing 
false edges from Fourier transformed image using notch filter, as discussed in Chapter 
3. In this scheme, based on analysis in frequency domain, it was found that blocking 
artifacts in decoded image appears periodic in the structure. These artifacts are expected 
to have periodic texture because block size is fixed in block coding but this kind of 
texture hides in the image and is difficult to detect them. However, these periodic- 
textures become more obvious after gradient operation as discontinuities due to block 
coding is highlighted in gradient image. According to signal processing theory, periodic 
texture will result in some peaks in frequency domain, which are repeated after regular 
spacing and corresponds to false edges due to blocking artifacts. 

This Chapter also deals with the design and applications of 2D multiple-notch filter 
but here the filter has finite impulse response (FIR). As discussed in Chapter 3, a notch 
filter highly attenuates a particular frequency and leaves the other relatively unchanged. 
So, a 2D multiple-notch filter can kill these peaks to reduce the artificial edges due to 
blocking artifacts in the gradient image. As there are multiple peaks in the frequency 
transformed (DCT coded) image, 2D multiple-notch filter is required to remove it. 
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Since this filter is used for reducing blocking artifacts from DCT coded images, it is 
meaningful to design multiple-notch filter with linear-phase which provides less distortion 
as compared to the filters with non-linear phase. A linear-phase is particularly important 
in image processing application as demonstrated in [64]. In this Chapter, we start with 
a review of design of ID FIR notch filter in section 4.3. Section 4.4 explains the design of 
2D multiple-notch filter. In section 4.5, design example is discussed. In section 4.6, two 
new applications of proposed filter are highlighted where as in section 4.7, the results 
have been presented to compare the proposed algorithm with the space- variant/ adaptive 
filtering algorithm. Finally, conclusions are given in section 4.8. 

4.3 Review of the Design of ID FIR Notch Filter 

Digital notch filters are the effective means for eliminating narrow-band or sinusoidal 
interferences in signal processing applications such as power line interference and can- 
cellation in electrocardiograms (EGG). A notch filter highly attenuates a particular 
frequency component in the input signal while leaving nearby frequency components 
relatively unchanged. Design of FIR digital notch filter is discussed in [9,50]. Yu et 
al. [9] investigated some approaches for designing ID linear-phase FIR notch filters, 
which are based on the modification of several established design techniques [65] of 
linear-phase FIR band selective filters. In this section, various existing techniques for 
designing linear-phase band selective filter with finite impulse response (FIR) filter are 
discussed. The commonly used methods for designing linear-phase FIR notch filter are 

(1) Windowed Fourier Series, 

(2) Frequency sampling approach, 

(3) Optimal FIR filter design approach. 

The aim of filter design is to make a reasonable dioice of the coefficients of the transfer 
function such that the response of resulting filter is a satisfactory approximation of the 
desired response. The notch filter is a very narrow band-stop filter. Ideally, bandwidth 
of notch filter should be zero with passband magnitude equal to unity and attenuation 
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Figure 4.1: Amplitude response of ideal linear-phase notch filter (a) Type 1 (b) Type 2 


at notch frequency should be infinite but in practice it has non-zero transition band 
and passband ripples. To provide the linear-phase, the notch filter must be a FIR filter 
with symmetric impulse response. Due to linear-phase requirement of FIR filter and the 
amplitude response requirements of notch filter, the length of a digital notch filter must 
be odd i.e. N = 2M + 1. The frequency response of a linear-phase FIR filter is given 

= (4.1) 

where A(6 j) is a real even function called the amplitude function to distinguish it from 
the magnitude function |i?(w)j and 0(w) is the phase function which is a linear function 
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of lo{9 = ujM). The linear-phase notch filter is completely characterized by A{u)) and its 
impulse response is a non causal sequence a(n) which is symmetric about origin. The 
causal sequence h{n) is given by 

h{n) = a(n - M) (4.2) 

Linear-phase notch filter can be divided into two types: Ideal Type 1 filter has a 180° 
phase shift at the notch frequency Wn as shown in Fig. 1 (a) whereas Type 2 linear-phase 
notch filter has exact linear-phase and there is no difference between magnitude function 
|i7(c(;)| and amplitude function A(a;) as shown in Fig. 1 (b). Magnitude function \H{oj)\ 
of both type of filter are same. 

The ideal Type 1 linear-phase notch filter is the amplitude-shifted version of an ideal 
lowpass filer. If A„(a)) is the amplitude response of ideal linear-phase notch filter, then 
amplitude function of corresponding low pass filter Ai{oj) are related with it as 

A(w) = ±[24(1*1) - 1] (4.3) 

The non causal response of two filters are related as 

On (n) = ±[2ai (n) - 5 (n)] (4.4) 

This relationship is based on the concept of complementary filter. The complementary 
filter transfer function G{z) is defined as 

H{z) = - Giz) (4.5) 

where iV— 1 is order of H{z) which is assumed to be even. ffH{z) is a Type 2 linear-phase 
notch filter with notch frequency at w„, G{z) is an extremely narrow-band bandpass filter 
called a tone filter. Thus design of a notch filter can be executed as the design of a tone 
filter and then converted to notch filter using above equation. 

4.3.1 Windowed Fourier Series design approach 

The frequency response of a linear-phase FIR filter, H{w), is a periodic function of oj 
with a period 2n. Thus, the impulse response of this filter is given by the Fourier series 
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coefficients of H{u}) of infinite length which leads to unrealizable filter. In this design 
method, the approximate version of H{(jS) is obtained by truncating the infinite impulse 
response to finite impulse response by windowing. The window functions are used to 
improve the convergence of the Fourier series in the vicinity of discontinuities where the 
well-known Gibbs’ phenomenon is present. 

Type 1 linear-phase notch filter 

Thus, if a,(n) are the Fourier series coefficients of the given ideal amplitude function 
Ai{ijj), then the coefficient are weighted by a selected windowing function w(n). 

a(n) = ai{n).w{n) (4-6) 

The amplitude function is given by 

A{uj) = (l/27r) r Ai{9)W{uj - 9)de (4.7) 

J—'K 

A desirable property of window function is that the function is of finite duration in the 
time domain and should also have most of the energy concentrated at low frequencies 
in its Fourier transform. Kaiser window function is very good approximation to this 
condition. Kaiser window is optimal window in the sense that it has the minimum 
spectral energy beyond some specified frequency. Thus, in windowed Fourier series 
approach, most frequently used window is Kaiser window and which can also be used 
for designing the FIR notch filter. Transition bandwidth and passband ripple determines 
the required length of the filter. Required passband ripples are obtained by the selection 
of window parameters. The filter attenuation is obtained by the choice of the parameter 
o; of the Kaiser window as 


[ 0.1102(A - 8.7), 50 < A 

0.5842(A - 21)°-4 -I- 0.07886(A - 21), 21< A < 50 

0, 7l < 21 


o; = < 


(4.8) 
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where A is the required ripple 5 of the lowpass filter in decibels 


A — —20 /o^io(^) (4-9) 

Kaiser also developed the relation between attenuation A and lowpass filter of length N 


N^{A- 7.95).27r/14.36Aa) (4.10) 

This formula is modified in [9] for linear-phase notch filter as 

N^{A- 1.93).27r/14.36Aa; (4.11) 

Type 2 linear-phase notch filter 


Type 2 linear-phase notch filter can be considered as an complement of an FIR tone 
filter. Ideal tone filter has very narrow rectangular frequency response Bi{u) centered 
at iwn and having bandwidth Ajo; tending to be zero. The convolution integral can be 
written as 


1 /'~Wn+(Aiw/2) \ ra)n+(Ajw/2) 

B(uj) = ^ / Bi(e)w{u - e)de + — / Bi{e)w{u - e)de ( 4 . 12 ) 

27r ./— Wn— (A{u;/2) 27r ./w„— (Aja;/2) 


c— Wn + (Aiw/2) 

-(Aiw/2) 

For small Ajo;, frequency response of tone filter can be written as 


B{u)) « (Aia;/27r)[Bt(u;„)IT(w - w„) + Bi{-LOn)W{u: 4- w^,)] (4-13) 

The normalization of tone filter response can be done using Bi(LOn) = Bi(— a;„) and the 
value B{ojn)- 

B{oj) /B (wh) = [W{u) — cOfi) 4- W(a; -f w„)]/[W (0) 4- R (Scj^)] (4-14) 

EVom this equation it can be seen that the tone filter response is equal to the shifted 
window spectra. The response of notch filter A(u)) can be given by 

A(a;) = 1 — [W(a; - a;„) + W{u; + Wn)]/{IF (0) 4- W (2a;„)] (4.15) 


The impulse response of the notch filter is then 


an(n) = 6{n) - 2bw{n)cos{o}nn) 


(4.16) 
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where b = 1/[W(0) + W'(2a;„)]. Thus, filter response is shifted window spectrum. The 
design of Type 2 linear-phase notch filter is reduced to the determination of the window 
with main lobe width equal to Aoo and the window ripple equal to 6. The formula 
is developed by Yu et al. in [9], for the design of Type 2 linear-phase notch filters 
using Kaiser window. The approximate formula giving relationship between a and the 
attenuation A is given by 






0.1284(A + 0.4551), 
0.1517(A- 4.7071), 
0.7798(A - 12.9164)°-4 
+0.0842(A- 12.9164), 



50.1943 < A 
20.5278 < A < 50.1943 

12.9164 < A < 20-5278 
A < 12.9164 


A practical formula for the required width derived by them is given by 


(A + 6.35)/11.87Ao; 


(4.17) 


(4.18) 


4.3.2 Frequency sampling design approach 

This is the most straightforward method of designing linear-phase FIR filter. The main 
idea of frequency sampling design approach is that a desired frequency response can be 
approximated by sampling it at N evenly spaced points and obtaining the interpolated 
frequency response that passes through the frequency samples. An FIR filter can 
be uniquely specified by giving either the impulse response h(n) or, equivalently, the 
Discrete Fourier Transform (DFT) coefficients H{k). DFT samples H{k) for an FIR 
sequence can be regarded as the samples of the filter’s z-transform, evaluated at N 
points equally spaced around the unit circle. Thus, amplitude response A(a;), specified 
at N equidistant points or frequency samples, gives the coefficients for the recursive FIR 
filter realization. It becomes computationally efficient method if coefficients Af- are zeros 
or ones. Therefore a typical lowpass filter consists of choosing A^ = 1 in the passband 
and Ajfc = 0 in the stopband and optimization of samples 0 < Aj, < 1 in the transition 
band. The optimization based on adjustment of aU samples results in optimum filter. 
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4.3.3 Optimal FIR filter design approach 

This method is one of the most powerful ID FIR digital filter design techniques proposed 
hy Park and McClellan, which is based on Remez exchange algorithm. These designs 
are based on the optimization of a, weighted Chebyshev error norm. McClellan et al. 
in 1973 [66] have developed useful program based on direct application of Chebyshev 
approximation theory to the linear-phase FIR filter design problem, and using Remez 
exchange algorithm. Main function of this program is to find equiripple behavior over the 
set of frequency bands, such as stopbands and passbands for band selective linear-phase 
FIR filters. Using this approach, an optimal linear-phase notch filter can be designed in 
two parts: first the realization of the infinite attenuation at the notch frequency with 
prefilter and second, the design of the corresponding equalizer which is cascaded with 
the prefilter to achieve the passbands equiripples characteristics. 


4.4 Design of 2D FIR Multiple-notch Filter 

The design of notch filter is discussed in [6,7,9,48-50,56]. In [9], various methods of the 
design of ID FIR notch are discussed. They have also discussed the design of 2D notch 
filter from ID notch filter. From the literature, it is difficult to find straightforward way 
to design two-dimensional filter. Several method of designing 2D FIR filter from ID FIR 
filter exists with less computing time but the resulting filter is not optimum, although 
they can be very good. First step is to approximate the frequency response of 2D filter 
as separable into two ID filters and then ID filter are designed using a suitable design 
technique. If the ID filters are optimum then 2D filter will also be optimal. Rabiner et 
al. in [67] proposed an approach for converting ID linear-phase FIR filter design into 
2D linear-phase filter designs. Design of FIR or nonrecursive filter involves determining 
a finite set of impulse response coefiicients such that filter characteristics satisfy the 
prescribed specifications. This is a problem of polynomial approximation. 

However, in this Chapter, a simple algebraic method is used to design a special 2D 
multiple-notch filter. In proposed method, which is based on [6], the 2D multiple-notch 
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Figure 4.2: Block diagram of a 2D FIR multiple-notch filter, 
filter is designed in following steps 

1. Two ID FIR multiple-tone (narrow band pass) filters are designed using well- 
known windowed Fourier series design approach. 

2. 2D multiple-parallel line filters are obtained from these ID FIR multiple-tone filters 
with their narrow pass-bands at appropriate frequencies based on the specifications 
of desired 2D multiple-notch filter. 

3. Cascading these two 2D multiple-parallel line filter gives a 2D FIR multiple-tone 
filter. 

4. The complementary filter of 2D linear-phase FIR multiple-tone filter is 2D linear- 
phase FIR multiple-notch filter. 

5. Finally, the notch at DC (zero frequency) of 2D multiple-notch filter is replaced 
by nearly all-pass filter to preserve the DC component of the image block, which 
contains the important information about the image. 

To design 2D multiple-notch filter, we start with the specifications of the desired 2D 
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Figure 4.3: (a) Impulse response of ID FIR multiple-notch filter (b) Frequency response 
of ID multiple-notch filter, (c) Amplitude response of horizontal multiple-parallel line 
filter, (d) Amplitude response of vertical multiple-parallel line filter, (e) Amplitude 
response of 2D FIR multiple-tone filter, (f) Amplitude response of 2D FIR multiple- 
notch filter. 
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multiple-notch filter, which are given by; 


[ 1, otherwise 


(4.19) 


where k=l,2, M; for M notch frequencies. For the design of such filter, a simple 

algebraic method is used. The specifications of corresponding 2D multiple-parallel line 
filters are; 

'l, forWi = ±u„, 

0, otherwise 

where i = 1,2 corresponds to vertically-parallel multiple-line filter and horizontally- 
parallel multiple-line filter, respectively. This kind of filters can be designed by taking; 




Hi{zi,Z 2 ) = Hmtf{z)\z=zi for i = l, 2 (4.21) 

where Hmtf{z) is ID multiple-tone filter and i = 1,2 corresponds to vertically and 
horizontally-parallel multiple-line filter, respectively. The well-known windowed Fourier 
series design approach is used to design ID multiple-tone filter. This approach is 
discussed in section 4-3. The most frequently used window function for FIR filter design 
is the Kaiser window, which is also used here to design linear-phase multiple-tone filter. 
The 2D multiple-tone filter can be obtained by cascading of these properly designed 
vertically-parallel multiple-line filter Hi{zuZ 2 ) and horizontally-parallel multiple-line 
filter 2^2 (zi, Z 2 ). The transfer function of 2D multiple-tone filter is given by: 

Ht{zi,Z2) = Hi{zi,Z2)H2{zi,Z2) (4.22) 

As in the case of ID linear-phase filters, a frequency-shifted window is a linear-phase 
FIR tone filter and its complimentary filter is a linear-phase notch filter. A(zi,Z 2 ) and 
B{zi, Z 2 ) are complementary [68] with respect to all pass function ^Z 2 ^ if 

A{Zi,Z2)+B{zi,Z2) = Zi^Z2^ (4.23) 

where 2M -|- 1 and 2Ar -f- 1 is the order of the two filter. The linear-phase 2D FIR 
multiple-notch filter can be obtained as the complementary filter of Ht{zi,z^. 

Hm{zu Z2) = - H,iz^, Z2) 


(4.24) 
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where iViand N 2 are the length of two ID naultiple-tone filter. The block diagram of 
2D multiple-notch filter is shown in Fig. 4.2 whereas its amplitude response is shown in 
Fig. 4.3 (f). 

4.5 Design Example 

In this section, a design example has been presented which was performed with MAT- 
LAB. 

Step 1: The notch frequencies are selected for the block size of 8 x 8 as: 


Wi = 0.257r 

BWi = 0.027r 

UJ2 = O.SOtt 

BW2 = 0.037r 

L 03 = 0.757r 

BW3 = 0.037r 

u )4 = 0.997r 

BWa = OMir 


These frequencies and bandwidths are also taken for the design of ID multiple-tone 
filter. 

Step 2: ID FIR multiple-tone filter is designed using MATLAB function with Kaiser 
window and we obtain the filter coefficients of 64*^^ order filter. The impulse response 
(filter coefficients) is shown in Fig. 4.3 (a) whereas the frequency response of ID multiple- 
tone filter is shown in Fig. 4.3 (b). 

Step 3: We obtain the transfer function of 2D multiple-tone filter Ht{zl,z2) by 
cascading 2D multiple-parallel line filters Hi{zl, z2) as shown in Fig. 4.2 . 

Step 4: The linear-phase 2D FIR multiple-notch filter can be obtained as the com- 
plementary filter of Ht{zl,z2). Amplitude response shows the notches at desired fre- 
quencies. 

Step 5: DCT coefficient of block size M x M is multiplied by 2D filter matrix of 
equal size. Where M is also the order of both 1-D filters. 
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4.6 Applications of 2D Multiple-notch Filter 

In [6] 2D HR single-notch filter is used for single or multiple sinusoidal-interferences. 
Proposed 2D multiple-notch filter can also be used for the removal of single or multiple 
sinusoidal-interferences. As discussed in Chapter S, 2D HR multiple-notch filter can be 
used for reducing blocking artifacts from the DCT coded image. Thus, 2D FIR multiple- 
notch filter can also be used for reducing blocking artifacts from the DCT coded image 
in the similar way as the 2D HR multiple-notch filter. These applications have been 
discussed below. 

4.6.1 Removal of false edges from the edge-map of blocky DCT 
coded image 

As discussed earlier, space- variant/ adaptive filtering requires a classification of image 
regions into different categories to apply a suitable filter in the corresponding region. 
Thus the performance of these algorithms is based on classification scheme which is 
based on edge information extracted from the received blocky image. Thus performance 
of these algorithms can be improved by removing the false edges from the edge-map 
of blocky DCT coded image. It is obvious from the property of DCT and it can also 
be observed that the DCT of square grid of line spacing of eight in both horizontal 
and vertical direction has peaks at an interval of 64 in both directions. This grid can 
be removed by killing these peaks by suitable notch filter. In this work this fact is 
verified as shown in Fig. 4.4 (a-d). The notch frequencies for this multiple-notch filter 
are calculated from block size. In case of actual images, it is observed that as the grid 
of lines in the edge image is not complete, the peaks remain at the same position but 
their spread increases. So, at a very high compression ratio, when grid in the edge 
map of DCT coded image is visible, this grid or false edges can be removed using this 
proposed 2D multiple-notch filter. This fact was verified with DCT coded 256 x 256 
Lenna image and was found to be correct as shown in Fig. 4.4 (e) and Fig. 4.4 (f ) . Thus, 
false edges from the edge-map of blocky DCT coded image can be removed by 2D FIR 
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Table 4.1: Comparison of the performances of various space- variant algorithms on Lenna 
image. 


Algorithms 

PSNR in dB at 
quality factor q 

Boundary 

PSNR in dB at 
quality factor q 

Boundary Disconti- 
nuity in fl0®j at 
quality factor q 

q=5 

q=10 

q=20 

q=5 

q=10 

q=20 

q=5 

q=10 

q=20 

Original 



- 

- 


- 

4.28 

4.28 

4.28 

DOT coded 

25.31 

28.09 

30.79 

15.0 

17.94 

20.93 

10.9 

7.95 

5.98 

Multiple- 
notch filter 
with SAF 

26.1 

28.84 

31.12 

16.49 

19.22 

21.42 

5.57 

4.28 

3.57 

Gaussian 

LPF 

with SAF 

26.1 

28.88 

31.20 

16.57 

19.32 

21.52 

5.23 

4.00 

3.33 


multiple-notch filter and the performance of space-variant/ adaptive filtering could be 
improved. 

4.6.2 Reduction of blocking artifacts by using multiple-notch 
filter in monotone areas 

In space- variant/adaptive filtering, the image regions are first classified into monotone 
area and edge area and then in the former low pass filtering is performed whereas in 
the latter directional or signal adaptive filter is applied. In proposed scheme, the same 
signal adaptive filter is applied in edge area but a 2D FIR multiple-notch filter is apphed 
in place of lowpass filter in monotone area. The 2D FIR multiple-notch filter is applied 
directly in monotone area because the periodic textures due to blocking artifacts axe 
more visible in these areas as compared to edge areas. The 2D FIR multiple-notch filter 
is not applied to the complete image as periodic texture due to blocking artifacts hide 
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(e) (t) 

Figure 4,5: Postprocessing of Lenna image at quality factor 5 in (a), (b), (c) and (d), (e), 
(f) at quality factor 10 by various algorithms. Here, (a) and (d) Before postprocessingj 
(b) and (e) Processed by 2D FIR multiple-notch filter with signal adaptive filter (SAF), 
Q^nd (c) and (f) Processed by Gaussian LPF with SAF 
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in DCT domain especially in edge area which is due to the masking effect of human 
visual system. Moreover, actual edges are also blurred if the same filter is applied in 
edge areas. In this scheme, spatial domain signal adaptive filtering algorithm is used in 
edge area to avoid blurring. The filtering mask for this two-dimensional signal adaptive 
filter is given by 

ABC 1/16 2/16 1/16 

DBF = 2/16 4/16 2/16 (4.25) 

G H E 1/16 2/16 1/16 

This filtering scheme is already discussed in Chapter 2. The results given in Fig. 4.5 
shows that blocking artifacts can be reduced by this method. However, the improvement 
is limited to high compression ratios because these periodic textures due to blocking 
artifacts hides in DCT domain at low compression ratios. 

4.7 Results 

The filter is designed according to the approach discussed in section 4-4 is imple- 
mented using MATLAB. The MATLAB functions are used for the design of ID FIR 
multiple-notch filter with Kaiser window. The block diagram of 2D FIR multiple-notch 
filter is shown in Fig. 4.2 whereas its amplitude response is shown in Fig. 4.3 (f). These 
plots of Fig. 4.3 show that all ID and 2D filters are obtained as per the design approach 
discussed in section 4.4. The 2D FIR multiple-notch filter has the notches at desired 
frequencies and the passbands ripples not very large. 

Now, in the rest of this section, the results of various algorithm applied on recon- 
structed blocky 256 x 256 Lenna image have been presented. These algorithms were 
implemented with MATLAB. Various results are given in Table 4.1. The results of two 
algorithms are compared with baseline JPEG like DCT decoded image. As the PSNR 
is only the rough indicator of image quality and does not reflects the blocking artifacts, 
two new discontinuity measures are defined. Various performance measures such as 
Peak SNR (PSNR), block boundary PSNR (BPSNR), block boundary discontinuity 
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have already been defined in Chapter 2. 

The reduction of discontinuity signifies that (see Table 4.1) blocking artifacts are 
reduced which can also be observed visually in Fig. 4.5. It can also be seen from Table 4.1 
that the improvement in PSNR for block boundary pixels is more as compared to over 
all PSNR. 

4.8 Conclusions 

In this Chapter, design and some new applications of 2D FIR multiple-notch filter are 
discussed. FIR filters are suitable for image processing applications as they causes less 
distortion due to their linear-phase property. The drawback of FIR filter is that they 
cause more delay. In this work, a special 2D FIR multiple-notch filter is designed with 
an intention to remove blocking artifacts from DCT coded images. For the design of this 
type of filter, first a 2D FIR multiple-tone filter is obtained from ID FIR multiple-tone 
filter. In proposed design approach, a simple algebraic method is used in which first a 
2D vertical and horizontal multiple-parallel line filters are obtained from ID multiple- 
tone filter and then a 2D FIR multiple-tone filter is obtained by cascading these two 
filters. Finally, the desired 2D multiple-notch filter is the complementary filter of 2D 
multiple-tone filter as shown in Fig. 4.2. Reduction of blocking artifacts and removal 
of false edges from edge-map of blocky image to improve the performance of space- 
variant/ adaptive algorithm are the new possible applications of 2D FIR multiple-notch 
filter that are considered in this work. Furthermore, the space-variant filtering method 
can be improved with a more efficient 2D FIR multiple-notch filter. 



Chapter 5 


Computationally Efficient 
Algorithms for Reducing Blocking 
Artifacts 


5.1 Introduction 

Most of the image compression standards use block discrete cosine transform (BDCT) m 
their image compression scheme due to its excellent energy compaction and decorrelation 
properties. DCT is used to exploit the spatial correlation for image compression. In 
BDCT based image compression, the image degradation becomes visible when compres- 
sion ratio exceeds certain level. These degradations manifest themselves as blocking 
artifacts due to the rigid block partitioning of the image and ringing noise around edges 
due to coarse quantization. Both effects are visually annoying and have substantial 
impact on the subjective quality of image. Blocking artifacts are visible because the 
inter-block correlation is not taken into account during encoding and quantization pro- 
cess of DCT coefficients. At higher compression ratios, since very few coefficients are 
encoded and each coefficient must be represented at a very coarse level of quantization, 
the blocking artifacts are more visible. 

Several postprocessing algorithms [W, 22-29) have been proposed to reduce blocking 
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effect of block coding system as discussed in Chapter 1. Out of these algorithms space- 
variant / adaptive filtering techniques are more attractive as they try to exploit the local 
statistics of the image. In these algorithms, the classification of image areas is used to 
capture the non-stationary behavior of the image. Classification is based on available 
edge information extracted from the received blocky image. Hence the performance of 
space-vaxiant or adaptive filtering scheme degrades. Furthermore, the need of strong 
filter in monotone area and week directional filter in edge area makes it complex. In 
references [2,4,5], the blocking artifacts are reduced by processing the image in DCT 
domain itself. 

In this Chapter ^ , four simple but effective algorithms for reducing blocking artifacts 
from DCT coded images are proposed. In original image, a large inter-block correlation 
exits between adjacent blocks and some amount of it remains in the blocky decoded 
image. To exploit the existence of large inter-block correlation between adjacent blocks, a 
signal adaptive filter is applied to DC component of DCT coded image. To further reduce 
the discontinuities, the image is re-divided into blocks in such a way that the comer of 
the original blocks comes at the center of new blocks. These discontinuities are reflected 
as AC component in the DCT of new blocks. The AC components of those blocks, which 
do not contain details, can be attenuated to reduce discontinuities. The performa 
of both the algorithms improve as compression ratio increases. These DCT dom 
algorithms are computationally efficient since fast algorithms for the computat’o 
DCT are available. In third algorithm, a second approach for discontinuity re 
is proposed which is based on SVD based filtering scheme. This algorithm i 
effective for reducing blocking artifacts. Fourth algorithm proposed in t ‘ 
again a DCT domain algorithm, which reduces the blocking artifacts by 
some DCT coefficients of concatenated block of two adjacent blocks. This algonthm has 
low computational complexity, as no filtering is required. In this Chapter, 
domain processing is discussed in section 5.2. Proposed four algorithms a 

Chapter repres^ts wo rk accepted for presentation and publication in [55] and 

communicated in [84] 
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in section 5.3. In section 5.4, the results have been presented. Finally, conclusions are 
given in section 5.5. 

5.2 Processing in Transform domain 

In DCT based image coding, the compressed image is in transform domain. Thus 
processing in DCT domain itself has low computational complexity. Following are the 
advantage of processing the image in DCT domain. 

1. To get processed image back in JPEG like format, IDCT-DCT calculation can 
be avoided. It can be shown that the computational complexity of filtering in 
DCT domain is significantly smaller than that of the straightforward approach, of 
converting back to the uncompressed domain, convolving in the spatial domain, 
and re transforming to the DCT domain. 

2. Distribution of noise in transform domain is random so its energy is spread over all 
the locations where as most of the signal energy is concentrated in a very narrow 
band, thus the effect of noise can be minimized. 

3. Quantization results in large number of DCT coefficients as zero. This property 
can be used to reduce the computational complexity. 

5.3 Proposed Algorithms for Blocking Artifact Re- 
duction 

Blocking artifacts are mainly due to independent coding of different blocks of an image. 
In original image a large inter-block correlation exits between blocks and some amount 
of it remains in the blocky decoded image. This residual inter-block correlation can be 
used for correcting the value of DC coefficient. In this Chapter, a signal adaptive filter is 
applied to sub-image of DC components of the DCT coded image. This filtering scheme is 
based on the idea that the DC components of blocks in monotone area are not changing 
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very rapidly and therefore in monotone areas, smoothing of DC components can be 
performed to exploit the residual inter-block correlation. The DC components of those 
blocks, which contain details, remain unchanged. This scheme reduces blocking artifacts 
without affecting much on peak signal to noise ratio (PSNR). The main advantage of 
this scheme is its low computational complexity. To further reduce the discontinuities 
in blocky image, the latter is divided into blocks in such a way that the corners of the 
original blocks come at the centers of new blocks. These discontinuities are reflected 
as AC component in the DCT of new blocks. The AC components of those blocks, 
which do not contain details, can be attenuated to reduce these discontinuities. Another 
advantage of this algorithm is that it also reduces the corner outliers. The performance 
of both the algorithms improve as compression ratio increases. 

Four algorithms are proposed in this section. In flrst subsection. Altering of DC 
component by signal adaptive filter is discussed whereas in second subsection the block- 
boundary discontinuity reduction algorithm is described. In the third subsection, a SVD 
bases discontinuity algorithm is discussed. Finally, in fourth subsection, another new 
DCT domain algorithm is discussed which reduces the blocking artifacts by eliminating 
some of the DCT coefficients of a concatenated block consisting of two adjacent blocks. 

5.3.1 ALGORITHM 1: Filtering of DC Component by Signal 
Adaptive Filter 

This algorithm is based on the assumption that the correlation among the DC coefficients 
of neighboring blocks are high if all the blocks belong to monotone area. In this 
algorithm, first a DC sub-image is constructed by taking only the DC coefficient of 
each block in DCT coded image and then a signal adaptive filter is applied to smoothen 
this DC sub-image. Filtering mask of this filter is shown in Fig. 5.1. Initial value of mask 
at center is taken as 0.75 whereas the value at other place is 0.0625. Experimentally 
it is found that these values are giving good results. The image blocks are classified as 
monotone block or edge block simply by comparing the absolute value of the sum of the 
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C 

D 


E 



Figure 5.1: Filtering mask of the Signal Adaptive Filter. 

AC coefficients of the block with a threshold. 

In this adaptive filtering scheme, if the center C of filtering mask is at DC component 
corresponding to edge block in DC sub-image then all surrounding coefficients A, B, D 
and E are made zero and this value remains unchanged. Since the blocking artifacts 
are more visible in monotone areas, it is assumed here that DC values corresponding 
to monotone block is more correlated with the DC value of neighboring blocks but it 
is not so for the DC value that corresponds to edge block. Thus, the DC components 
corresponding to edge blocks are excluded from the filtering. If the center of mask is at 
DC component corresponding to monotone block, then those surrounding values that 
correspond to DC component of edge blodcs are decreased to zero whereas weight of 
center of mask is increased by the same amount. These filtered values of DC coefficients 
are fed in original image blocks and then inverse DCT is performed to reconstruct the 
image. This scheme shows significant reduction in blocking artifacts without much com- 
putation complexity. This can be verified by calculating block boundary discontinuity. 
The PSNR is slightly improved but the improvement is more at block boundary pixels. 
These results are given in Table 5.1. 
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Figure 5.2: Original (solid line) and new position (dotted line) of blocks used for reducing 
discontinuities in algorithm 2 and 3. 

5.3.2 ALGORITHM 2: Reduction of Block Boundary Discon- 
tinuities 

Proposed algorithm is based on the observation that artificial discontinuities at block 
boundaries are due to blocking artifacts and are more visible in monotone or slowly 
varying areas. Thus, this algorithm is also applied m monotone areas. The blocky image 
is re-divided into new blocks such that it captures the maximum artificial discontinuities. 
In order to reduce discontinuities, the AC components corresponding to discontinuities 
axe identified and then selected AC components of those new blocks, which do not contain 
details, are heavily attenuated. Although this algorithm requires the computation of 
DCT of new block, the processing time is less because no filtering is required. Availability 
of fast algorithms for DCT computation further reduces the processing time. This 
algorithm has following steps 

1) Division of blocky image into new blocks 

The blocky image is re-divided into blocks in such a way that the comer of the original 
blocks comes at the center of new blocks as shown in Fig. 5.2. This type of re-division 
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captures the maximum artificial discontinuities in new blocks. These discontinuities are 
reflected as AC component in the DCT of new blocks. 

2) Classification of blocks 

To protect actual edges, the discontinuity algorithm is applied to monotone blocks only. 
Thus, the classification of new blocks into edge block and monotone block is required. 
Classification of block can be performed in DCT domain by examining the values of 
DCT coefficients. In the proposed algorithm, a DCT based block classification scheme 
is used in which the sum of absolute values of AC components is used to classify the 
blocks into monotone block and edge block. The average of sum of absolute values of 
AC components for all blocks is taken as the threshold for the classification. A block is 
considered as monotone block if the sum of absolute values of AC components for that 
block is below the threshold value. 

3) Discontinuity Reduction 

Prom the formula of DCT it can be shown that when only the DCT coefficients in the top 
row axe non-zero then all the pixels in each column has equal values in spatial domain or 
vise-versa. Similarly, if the DCT coefficients in the first column are non-zero then all the 
pixels in each row have equal values in spatial domain. These facts and trends support 
that monotone blocks having artificial edges have higher values in first row or column [3] 
because they may have vertical, horizontal or both t 5 q)es of edges/discontinuities due 
to blocking artifacts. The spatial block f = {fij} can be transformed to the frequency 
(DCT) block F = {F^,} as: 


N-l N-1 ^ 

F.. = S E for 0<u,v<{N-l) (5.1) 

i=0 i=0 


Where 




(5.2) 


Where = 2 /N for u 7 ^ 0 and v 7 ^ 0 ; = y/2/N for either u = 0 or v = 0 ; and 

Cuv = 1/N for both u = 0 and v = 0. Here Fqo is DC component whereas vertical (Fv), 
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horizontal (Fh) and remaining AG components (F^c) are: 


Fv = [Foi,Fo 2, Fo(Ar_i)] 

Fh = [Fyo, F20, F(Ar_i)o]^ 

Fac = {Fuj,, N — 1 > U,V > (5-3) 

Quantized and decoded block can be written as: 

Ar-l N-\ 

/ii= E E Fii D'fy (5.4) 

i=0 j=0 


r\j 

Where Fuv is the quantized DOT coefficient of 

The blocking effect can be estimated for monotone blocks using transform coefficients 
as they are estimated in [2]. Let current, left, right, top and bottom blocks are denoted 
by C, L, R, T and B respectively. Blocking effect occurs along the boundary of successive 
smooth blocks in the spatial domain, thus vertical blocking effect can be estimated as: 


4''* = (E I')'" 

j=Q 

Where is the boundary difference given by: 

/ r \ ^ (^) W 


(5.5) 


(5.6) 


This boundary difference in DCT domain can be given by: 


B 


[L] 


N-l 

X) X) 

it =:0 t ;=0 


iC) 


^jO 


N'-liV-l 

EEC. 

u=0 v=0 


(i) 




(6.7) 


The boundary difference in DCT domain can be divided into DC, V, H, AC components. 
The contributions of these components to vertical boundary difference is discussed in [2]. 
We can draw following conclusions to apply the results in proposed algorithm: 


(a) Boundary difference contributed by DC component depends on the difference of 
DC levels of two adjacent blocks. Since we have already minimized this component 
by first algorithm, so we can neglect it. 



Chap. 5 Computationally Efficient Algorithms for... 


100 


(b) Boundary difference contributed by V component is major part that greatly affects 
the blocking effect along the vertical direction. 

(c) Boundary difference contributed by H and AC components produce high frequency 
variations along the vertical direction and results in perceptible invisible blocking 
effects. So they can be neglected. 

(d) Similarly it can be shown that boundary difference contributed by H component is 
a major part that greatly affects the blocking effect along the horizontal direction. 

In proposed algorithm, based on the above discussion, the DCT coefficients of first 
two rows and columns of monotone blocks are heavily attenuated whereas the DCT 
coefficients of edge blocks are less attenuated to reduce artificial discontinuities. Here 
second row and second column are also selected for attenuation because experimen- 
tally we found that vertical and horizontal discontinuities in blocky image have strong 
showings in first two columns and rows, respectively, in DCT domain. 

5.3.3 ALGORITHM 3: Reduction of Block Boundary Discon- 
tinuities: Second Approach 

In this algorithm the first two-steps i.e. division of blocky image into new blocks and 
classification of blocks remain the same but the third step i.e. the approach for the 
reduction of discontinuity is different. The second approach of discontinuity reduction 
is based on block-based, non-linear filtering algorithm. This filtering approach is based 
on singular value decomposition (SVD) algorithm. Linear filtering is used to remove 
the additive noise but their major drawback is that they tend to blur the edges. Non- 
linear filtering based on SVD method has been successfully used in past for various 
noise removal applications [69-71]. This technique does not require any additional o 
priori information. It is well known that random noise is hard to compress whereas 
ordered information is not. Thus, lossy compression of noisy signal, with allowed loss set 
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equal to the noise strength provides the required filtering of the signal. Proposed SVD- 
based filtering can effectively reduce the discontinuities while preserving the edge details. 
SVD algorithm can be viewed as a lossy compression scheme. For monotone blocks, 
the discontinuities are hard to compress as compared to smooth area therefore SVD 
algorithm allows to filter out these discontinuities. Derivation of the optimum threshold 
to distinguish significant and insignificant singular values for reducing discontinuity is 
still a problem with this approach, which must be investigated. The threshold calculation 
requires the additional signal strength due to discontinuities. The theory of SVD-based 
filtering is discussed below. 

Any m x n real- valued matrix A can be decomposed uniquely as 

A = UT.V'^ = Yh oiiUjvf (5.8) 

1=0 

where U and V are orthogonal matrices with column vectors Ui and v, and S = 
diag(a!i,Q;- 2 ,...,tt„) is a diagonal matrix. The diagonal elements of E can be arranged in 
descending order and are called the singular values of A. Thus, SVD provides the rank 
r of a matrix, which is equal to the number of non-zero singular values of the matrix. If 
rank of the matrix A is r then 

Oll>Ci2> — > > 0!r+l = ... = Q!n = 0 (^-9) 

For additive noise model, observation matrix B is equal to A + E, where E is a random 
noise matrix of full rank. So, the last (n-r) singular values of B are small but not 
necessarily zero. 

Let B he an M X N image divided into k x k image subblocks Bk- Each subblock 
can be expressed as weighted sura of basis images and weights are the singular values. 
Setting “nonsignificant” singular values to zero results in lossy compression. The amount 
of loss is controlled by a threshold value e. The effectiveness of algorithm depends upon 
the value of e. If e is very small then most of the noise remains but if e is very large then 
image information is lost. Thus, the main steps of SVD-based filtering algorithm are 

(1) Divide image B into subblocks Bk- 
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(2) Apply SVD to each subbiock and set the singular values less than a threshold e to 
zero. 

Bfc = UEV'^ = OiUivf (5.10) 

i=0 

If E' is resultant diagonal matrix of singular value after threshold then the recon- 
structed block will be 

B'k = U'E'V'^ = ^ aiUivJ (5.11) 

t=0 

(3) Replace Bk with where is resultant block after setting the singular values 
less than a threshold e to zero. 

(4) Apply step (2) an (3) to all the blocks and reconstruct the image. 

Note that this algorithm is applied to new blocks obtained by re-division of blocky 
image into block as discussed in step 1 of discontinuity reduction algorithm. It is found 
that the reduction in blocking artifacts cannot be achieved with the original blocks. This 
approach reduces the blocking artifacts more efficiently but its computational complexity 
is more because SVD algorithm has to be applied to each of the block. 

5.3.4 ALGORITHM 4: Blocking artifact reduction by elimi- 
nating some DCT coefficients of concatenated block of 
two adjacent blocks 

As we know that edge component causes the high frequency components in the trans- 
formed domain. Thus, examining the DCT coefficients of the block itself can identify the 
monotone blocks. As discussed in Chapter 1, the discontinuities in monotone area are 
due to blocking artifacts. These discontinuities also cause the high frequency components 
in DCT domain. Thus, these discontinuities are reflected as high frequency components 
in the DCT of concatenated block of two or more neighboring monotone blocks. It is 
also found that there exists a relationship between the DCT coefficient of two monotone 
blocks and that of concatenated block. The DCT coefficients due to blocking artifacts 
afe present in the concatenated block but not in individual block. Therefore, from 
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the relationship between the DCT coefficients of monotone block of different sizes we 
can detect the high frequency components due to blocking artifacts and thus we can 
eliminate them to reduce the blocking artifacts. It is well known that N-point DCT 
C{k) is closely related with the 2N-point discrete Fourier Transform (DFT) F{k) for 
symmetrically extended sequence. 

C{k) = (5 

To find the relationship between N x N 2D DCT and 2N x N 2D DCT we extend 
the results obtained in [4,39]. The two adjacent rows u{n) and v{n) are concatenated 
as w{n) and their DCT are given by U{k), V{k) and W{k), respectively, as 

U{k) = a{k) u{n) cos[^ ^”^ (5.13) 

F {k) = a{k) v{n) cos[- ^ -- | ^ ^^^ ] (5.14) 

W{k) = E (5.15) 

where a{k) = for A: = 0 and q:(A:) = for other values. W{k) can be expressed 
as 

[ jsW{km + (-l)'‘/’V(k/2)], for t = 0,2,4, ... 

«-(<:)= (5.16) 

+(_l)M/2„(„)cosl!;!^]} for * = 1,3,5,... 

Prom the above equation it is clear that the 2*"' DCT coefBcient of depends only 
on the k*^ DCT coefficients of U and V whereas the odd numbered DCT coefficients of 
W are expressed as a weighted sum of u{n) and v{n). Thus, only the odd numbered 
DCT coefficients of W are affected by the discontinuities in the block. Thus, actual 
nonzero coefficients due to signal are kept within the range two times larger than those 
of U and V. For concatenated 2D blocks, actual nonzero DCT coefficients are present 
within the range two times larger than those of DCT coefficients of constituent block 
for all rows and column. The coefficients beyond this range are the coefficients due 
to the blocking artifacts. In proposed algorithm, the DCT coefficients in the first row 
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and first cohnnn that are beyond this range are eliminated. Only first row and first 
column are selected because of the effect of vertical and horizontal discontinuities which 
has greater values in these locations as discussed in subsection 5.3.2. This algorithm 
has low computational complexity, as no filtering is required. The availability of fast 
algorithm for DOT computation makes this algorithm suitable for low bit rate coding 
applications. The selection of first row and first column for elimination further reduces 
the computation time. Following are the main steps of this algorithm. 

Step 1. Identification of block as monotone block: Actual edges are blurred due 
to elimination of high frequency components. To protect the actual edges, this 
algorithm requires the identification of monotone blocks. In proposed algorithm, 
we selected only first row and first column for elimination of some DCT coefficients. 
Therefore, this step can be bypassed. In fact, in implementation of this algorithm, 
this step is ignored because this algorithm doesn’t produces much undesirable blur 
as discussed above. 

Step 2. Joining two horizontally adjacent monotone block and transforming 
the concatenated block in DCT domain: The two horizontally adjacent 
monotone block of size M x M Bxe concatenated into one block of size 2M x M. 
The DCT coefficients due blocking artifacts are present in the concatenated block 
but not in individual block. The 2M x M DCT of concatenated block gives the 
global characteristics. 

Step 3. Finding the maximum non-zero valued index for the concatenated 
block for elimination of DCT coefficients corresponding to blocking 
artifacts: The global frequency characteristics in two adjacent blocks are similar 
to local ones in each block. This concept is used in identifying and eliminating the 
coefficients due to blocking artifacts in the concatenated block. For two blocks, the 
index of first row and first column are found by checking them for non-zero values. 
The maximum non-zero valued index for any row or column of a block is defined 
3s the location beyond which all the DCT coefficients are zero. For concatenated 
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5.4 Results 

In this section, results of various algorithm applied on 256 x 256 Lenna image are 
presented. These algorithms are implemented with IMATLAB. To get blocky image 
at different quality factors, a baseline JPEG like algorithm is implemented and then 
this blocky image is postprocessed by various algorithms. Various results are given in 
Table 5.1. Various quality measures are calculated to see the performances of these 
algorithms. The results of two algorithms are compared with baseline JPEG like DOT 
decoded image. Peak SNR (PSNR) is defined in Chapter 2. Subjective quality of an 
image depends on the properties of human visual system (HVS), as discontinuities are 
more visible in monotone or slowly varying areas. Therefore, the PSNR is only the 
rough indicator of image quality and does not reflect the blocking artifacts. Two new 
discontinuity measures are defined as discussed in Chapter 2. The block boundary 
PSNR (BPSNR) is defined in the same way as PSNR but only one pixel from both side 
of block boundary are considered for the calculation of MSE whereas the block boundary 
discontinuity measure is defined as the sum of square of difference of pixel values across 
the four block boundaries. 

Filtering of sub-image consisting of DC components by signal adaptive filter reduces 
blocking artifacts with very low computational cost whereas the selective attenuation of 
AC component corresponding to block boundary discontinuities shows significant reduc- 
tion in blocking artifacts. Finally corner outliers detection and replacement algorithm 
is applied to further improve the performance. The performance of both the algorithms 
improves as compression ratio increases. The reduction of discontinuity signifies that 
(see Table 5.1) blocking artifacts are reduced and it can also be observed visually in 
Fig. 5.3 and Fig. 5.4. It can also be seen from Table 5.1 that the improvement in PSNR 
for block boundary pixels is more as compared to over all PSNR. 

The second approach for discontinuity reduction is proposed in algorithm 4. It 
reduces the blocking artifacts more efficiently but its computational complexity is more 
because SVD algorithm is to be applied to each of the block. PSNR achieved by this 
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Figure 5.3: Postprocessing of Lenna image compressed at quality factor of 5 by various 
algorithms, (a) Before postprocessing, (b) DC component filtered, (c) Discontinuity 
reduction, (d) Corner outliers removed. 
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(c) (d) 


Figure 5.4: Postprocessing of Lenna image compressed at quality factor of 9 by various 
algorithms, (a) Before postprocessing, (b) DC component filtering by algorithm 1. (c) 
Discontinuity reduction by algorithm 2. (d) Corner outliers removed. 
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Figure 5.5; Postprocessing of Lenna image compressed at qualty factor of 5 in (a), (b) 
and (c) and 9 in (d) , (e) and (f) by various algorithms, (a) and (d) Before postprocessrng. 
(b) and (e) Discontinuity reduction by algorithm 2. (c) and (f) Discontrnmty reduction 


hv svn filtering ( AlgaiiamUl 
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Figure 5.7: Poetproceu.uing of Lenna image compressed at quaUty factor of 9 b, various 
algorithms, (a) Before postprocessmg. (b) Discoutinuity reduction by algont 
Algorithm 4. (d) Algorithm 4 with corner outliers removed. 
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Table 5.1; Comparison of the performances of proposed computationally efficient 
algorithms on Leima Image. 


Algorithms 

PSNR in dB at 
quality factor q 

Boundary 

PSNR in dB at 
quality factor q 

Boundary Discon- 
tinuity (in 10® jot 
quality factor q 

q=5 

q=9 

q=12 

q=5 

q=9 

q=12 

q=5 

q=9 

q=12 

Original 

- 



- 

- 

- 

4.28 

4.28 

4.28 

DCT coded 

25.32 

27.71 

28.80 

15.02 

17.54 

18.71 

10.9 

8.20 

7.19 

DC component 
filtering: 

Algorithm 1 

25.42 

27.71 

28.82 

15.13 

17.56 

18.73 

10.4 

8.06 

7.13 

Discontinuity 

reduction: 

Algorithm 2 

25.60 

27.87 

28.95 

15.45 

17.85 

18.95 

9.47 

7.46 

6.71 

Algorithm 2 
with comer 
outliers reduction 

25.75 

27.98 

29.00 

15.89 

18.20 

19.15 

7.74 

6.19 

5.69 

Discontinuity 
reduction: 
Algorithm S 

25.45 

27.71 

28.77 

15.26 

17.74 

18.82 

9.95 

7.58 

6.76 

Algorithm ,? 
with comer 
outliers reduction 

25.60 

27.81 

28.80 

15.71 

18.08 

19.01 

8.22 

6.37 

6.12 

Blocking Artifact 

reduction: 

Algorithm 

25.55 

27.79 

28.85 

15.29 

17.63 

18.77 

9.58 

7.87 

7.00 

Algorithm f 
with comer 

L outliers reduction 

25.71 

27.91 

28.91 

15.76 

18.01 

18.97 

7.87 

6.55 

5.96 
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algorithm is also slifthtly h-s^ as compared to the first approach. The results from this 
algorithm are presented in 'Fahle 5.1 and Fig. 5.5. 

Finally, the fmirth algorithm proposed in this Chapter is also implemented with 
MATLAB. This rompntationally efficient algorithm also gives very good result in terms 
of PSNR, discontinuity measure and visual quality. The performance and results of this 
algorithm are present eel in Fig. 5.6, Fig. 5.7 and Table 5.1. It is clear from these results 
that this algorithm also gives very good performance with minimum computational 
complexity. 

5.5 Conclusions 

In this Chapter, fotir simple y<'t <*lfertive algorithms for reducing blocking artifacts from 
DCT coded image are proposed. Although these algorithms look very simple but they are 
able to r<xluc(* blocking artifacts significantly without much computational complexity. 
The performance of tlu'se algorithms is better at high compression ratios; therefore 
they are suitable for low bit rate video coding applications. In first algorithm the signal 
adaptive filter is applied to sub- image consisting of DC components of DCT coded image. 
This algorithm reduces the blocking artifacts with very low computational complexity. In 
second algorithm the blocking aartifacts are reduced significantly by selective attenuation 
of AC components corresponding to blodk boundary discontinuities. These results axe 
verified by comparing botiiulary lASNR and block-boundary discontinuity (see Table 5.1). 
This can also be ()bserve<l from the results presented in Fig. 5.3 and Fig. 5.4. It is found 
that improvement in FSNR for block boundary pixels is more as compared to over all 
PSNR. This result is attractive because discontinuities at block boundaries, which are 
the most visually annoying effect, are reduced. Hence, these computationally efficient 
schemes reduce blocking artifacts considerably with slight improvement in PSNR. 

ffi third algorithm proposed in this Chapter, the discontinuity reduction is based on 
SVD-based filtering scheme. It reduces the blocking artifacts more efficiently but its 
computational complexity is more because SVD algorithm has to be applied to each of 
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the block. This algorithm is very effective for reducing blocking artifacts but PSNR 
achieved by this algorithm is also slightly less as compared to the first approach. 

Fourth algorithm proposed in this Chapter is again a DCT domain algorithm, which 
reduces the blocking artifacts by eliminating some DCT coefficients of concatenated 
block of two adjacent blocks. This algorithm has low computational complexity, as no 
filtering is required. The availability of fast algorithm for DCT computation makes this 
algorithm suitable for low bit rate coding applications. The selection of first row and first 
column for elimination further reduces the computation time. The results show that this 
algorithm provides very good performance with minimum computational complexity. 


Chapter 6 


Conclusions and Future scope 


6.1 General 

In this Chapter we summarize the results obtained in this thesis and present our con- 
clusions along with the suggestions for research in the area of DCT based coding and 
postprocessing. In this work, a number of postprocessing algorithms are proposed. All 
these algorithms are implemented in MATLAB and some of these are also implemented 
in C. The reason for using MATLAB was to save the time in writing the long code 
for implementation of the various algorithms. The various functions available in image 
processing toolbox are used. First, to make the platform for the work, a simple baseline 
JPEG like algorithm for Imy image compression is implemented. Almost all the stages 
of this algorithm i.e. dividing the image into blocks, taking DCT of each block, their 
quantization and inverse quantization at different quality factors, IDCT and reconstruc- 
tion of image, are implemented. Some more programs are written for finding the number 
of bits required to encode the quantized image and the calculation of various performance 
measures such as PSNR, boundary PSNR and block-boundary discontinuity. With this 
preliminary work, all of the proposed algorithms are implemented. To get decompressed 
blocky image at different quality factors, a basehne JPEG like algorithm is implemented 
mid this image is postprocessed by various algorithms. The results of various algorithms 
proposed in this work are already given at the end of the each Chapter. 
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6.2 Sumniary and Main findings 

In this thesis work, a number of algorithms for postprocessing of DCT coded images are 
proposed. These adgorithms are implemented with MATLAB and they are applied to 
process the decompressed Lenna image to compare the performances. The performances 
of these algorithms are verified by comparing the PSNR, boundary PSNR and block- 
boundary discontinuity with baseline JPEG like scheme. The corner outliers reduction 
scheme improves the performance of all proposed algorithms. In this section, these 
algorithms are summarized. 

In Chapter S we proposed a hybrid scheme for the postprocessing of DCT coded 
images. This scheme is named as hybrid as the processing is performed in both DCT as 
well as in the spatial domain by two different algorithms. This algorithm includes the 
advantage of adaptive filtering in edge area and block-boundary discontinuity minimiza- 
tion algorithm in monotone area. Thus, the processing of image is performed in both 
spatial and transform domain. In this algorithm, a lower order signal adaptive filter is 
applied in edge area to reduce the processing time. This scheme uses the block-boundary 
discontinuity minimization algorithm in monotone area, which also does not have much 
computational cost. Corner outliers reduction scheme further improves the performance. 

In this scheme, the DCT coefficients of monotone block are compensated to achieve 
minimum block-boundary discontinuity. These compensating DCT coefficients for mono- 
tone blocks are calculated from the DCT of the block-boundary pixel differences. In this 
calculation, those block-boundary pixel differences are made zero whose neighboring 
block is an edge block. In proposed scheme, the edge information is extracted and used 
for the classification of image blocks. For edge blocks, a signal adaptive filter is applied 
to perform the filtering along the edge. This filter also uses the same edge information 
(edge-map) to perform the filtering approximately parallel to edge to avoid blurring. It 
has been found that the performance of the proposed algorithm improves as compression 
ratio increases. The reduction of discontinuity signifies that the blocking artifacts are 
reduced. The improvement in PSNR for block-boundary pixels is more as compared to 
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over all PSNH, which also indicates the same. It has also been observed that neither of 
the two alKorithnis used in the hybrid scheme are able to give good results if they are 
used to process th(‘ wlioh‘ image. 

In Chapitr H we dt'veloijed the design method for 2D HR multiple-notch filter and 
some new applications are being proposed. In proposed design approach, a simple 
algebraic method is user! to design the 2D HR multiple-notch filter. In this approach, 
first two ID multiple-tone filters are designed as per the specifications of 2D multiple- 
notch filter and then these two filters are used to obtain 2D multiple-vertical line filter 
and 2D multiple-horizontal line filter. Finally, cascade of these two filters gives the 
desired 2D mult iide-notch filter. The design of ID multiple tone filter is based on the 
design of all-pass filter as discussed in [7]. 

Then, a (hisign exami>le and new applications of 2D multiple-notch filter are dis- 
cussed. It is shown i haf 2D multiple-notch filter can eliminate the square grid. It can also 
reduce the blocking artifacts from DCT coded images. The concept of reducing blocking 
artifacts by 2D multiple-notch filter is based on the fact that the discontinuities due to 
block artifacts in block decoded image have periodic structure which ultimately results 
in some peaks in the DCT domain. These discontinuities are highlighted in gradient of 
the image. A 2D multiple-notch filter can kill these peaks to reduce blocking artifacts. 
Discontinuities due to blocking artifacts axe more visible in the monotone area, so this 
filter can be applied directly in th^e areas. Thus, improvement in the performances of 
space- variant/aw^laptive filter by the removal of false edges from the edge-map of blocky 
image and the reduction of blocking artifacts directly in monotone area, are the new 
possible applications of 2D multiple-notch filter. In these applications, the reduction of 
boundary PSNR and block-boundary discontinuity signifies that the blocking artifacts 
are reduced. 

In Chapter 4 we extended the design approach of 2D multiple-notch filter to FIR 
type filter. FIR filters axe suitable for image processing applications as they cause less 
distortion due to their linear phase property. The drawback of FIR filter is that they 
cause more delay. In this work, a special 2D FIR multiple-notch filter is designed with 
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an intenlioii rciiun’c hlnckiiig artifacts from DCT coded images. 

The design pr< >re<inre f. >r t he 2D FIR multiple-notch filter is similar to the procedure 
that is used for 11 h> df‘sign of 21) HR multiple-notch filter. However, the desired 2D 
multiple-notch tilter is a compleimuitary filter of 2D multiple-tone filter. In this case, 
two ID FIR mult ipl(‘-t one fillers are designed by windowed Fourier series method with 
Kaiser window. I'he filter is designed and implemented using MATLAB. Various plots 
show that all ID and 2D filters are obtained as per the design approach. The 2D FIR 
multiple-notch filter has the notdhes at desired frequencies and the passband ripples are 
also not very large, rhis filter is also applied for the reduction of blocking artifacts and 
removal of false edgfv from edge of !>locky decompressed image. The filter is applied 
on decorapresse<l hlorky Leima image and the comparison of performance and visual 
observations signifies that blot'king artifacts are reduced. 

In Chapter 5, fotir simple and effective algorithms for reducing blocking artifacts from 
DCT coded images are being propos«Hi. Although these algorithms look very simple but 
they are able to r(‘duce blocking artifacts significantly without much computational 
complexity. 'The performance of these algorithms is better at high compression ratios; 
therefore these are suitable for low bit rate video coding applications. 

First algorithm proposed in Chapter 5 reduces the blocking artifacts by exploiting 
the residual inter- block correlation, which exists in the received blocky image. For this 
purpose a signal adaptive filter is use<l to smoothen out a sub-image of DC coefficients 
of all the blocks. 'I'liis algorithm reduces the blocking artifacts with a very low computa- 
tional complexity. S«H'<>iui algorithm reduces the block-boundary discontinuities due to 
blocking artiffU'.ts by capturing and removing them in the DCT domain. The blocking 
artifacts are rwluced significantly by selective attenuation of AC components due to block 
boundary discontinuities. These two algorithms together reduce the blocking artifacts 
considerably. It has been verified by comparing boundary PSNR and block-boundary 
discontinuity. It has been found that improvement in PSNR for block boundary pixels is 
“lore as compared to over all PSNR. This result is attractive because discontinuities at 
block boundaries, which are the most visually annoying effect, are reduced. Hence, this 
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computationally efficient schei.ie reduces the blocking artifacts considerably with slight 
improvement in PSNH. In the third algorithm, a second approach for discontinuity 
reduction is i>roposed winch uses the SVD based filtering scheme. In this algorithm, the 
block-boundary dis(’onf inuities due to blocking artifacts are captured in the same way 
but the SVD based filtering .scheme is used for their reduction. It reduces the blocking 
artifacts considerably hut its computational complexity is slightly more because SVD 
algorithm has to Ire applied to eac.h monotone block. The PSNR achieved by this 
algorithm is also slightly less as compared to the first approach. 

Fourth algorithm proposer! in Chapter 5 is again a DOT domain algorithm. It reduces 
the blocking artifacts by eliminating some selected DCT coefficients of concatenated 
block of two arijacent blocks. 'Phis algorithm has low computational complexity, as no 
filtering is required. 'Fhe availability of fast algorithm for DCT computation makes 
this algorithm suitabh* for low bit rate coding applications. In this algorithm, only 
the first row and first column are selected for elimination which further reduces the 
computation time, 'fhe results show that this algorithm provides very good performance 
with minimum computational complexity. 

6.3 Conclusions 

DCT based image compri’ssion is very popular and found various applications till date, 
thus efficient postproc«s.sing of such image has become an important research topic. This 
thesis addresses various postprocevSsing issues that may arise in practice: (i) reduction of 
coding artifacts; and (ii) computationally efficient methods to make algorithm suitable 
for real time applications such as video and still image decoding. Main finding of this 
thesis is that the coding artifacts can be reduced efficiently by postprocessing. At higher 
compression ratios, the coding artifacts are more visible and they can be reduced more 
efficiently than the existing postprocessing algorithms. Various aspects of DCT based 
image compression and post processing methods axe studied with a state-of-art survey 
00 the subject. A number of new postprocessing algorithms for reducing the coding 
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artifacts arc proposfni in this work. 

The first rontrihuf i(»n of the th(*sis is to develop a hybrid scheme for postprocessing 
by clubbing the good things of several existing algorithms. Smoothing artificial discon- 
tinuities due t(» blocking artifacts improves the image quality whereas smoothing actual 
image edges degra<ies it . I hns, objective was to satisfy these two conflicting requirements 
as simply as possible. Propose<} hybrid scheme takes advantages of adaptive filtering and 
a method for the recovery of loss of accuracy of DCT coefficients. In this algorithm, 
no filtering is recpiire*! in monotone area but only the computation of compensating 
coefficients is reeprired. I’erformance of this algorithm is better than space-variant 
filtering. This scheme rminr*^ !)!ocking artifacts considerably without introducing much 
blur in edge areas and also it provide slight improvement in PSNR. 

The secoml contriluition of the thesis is the design of 2D multiple-notch filter and 
some of its new aj){)lirations in rwlucing blocking artifacts. The design method for both 
FIR and IIH filter of this type are (ieveloped. The possibility of using this filter for 
reducing blocking artifact is fully examined. For this purpose, this filter is applied to 
improve the perfonnance of the space- variant/adaptive filtering method. It is found that 
the performance of space-variant filtering can be improved by including the proposed 
2D multiple-notch filter for false edge removal in ite classification scheme. Reduction 
of the blocking artifacts directly in monotone area is the second application suggested 
for this filter. The application of 2D multiple-notch filter for reducing blocking artifact 
is new and it is found that the performance of the space-variant algorithm utilizing 
2D multiple-not,ch filt(m are comparable to existing algorithms. Thus, 2D multiple- 
notch filter has been successfully applied for reducing the blocking artifacts. To reduce 
the computational complexity, the filtering is performed in DCT domain. The results 
obtained in case of FIR filter are better than HR filter, as expected. 

The third contribution of the thesis is to provide four computationally efficient 
algorithms for postprocessing of DCT coded images. In literature several powerful 
postproc^ing algorithms have been developed to reduce blocking artifacts but even a 
^nry simple algorithm can be too complex for real time applications such as video and still 
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image decoding. 1 hose Minplo Hlgorilhins can improve visual quality with improvement 
inPSNR and are enmpef dnv f«» thi. many deblocking algorithms reported in literature. 
Four algorithms are piopow^i m Chuptfr f,. First algorithm reduces the blocking artifacts 
by exploiting tin* resnlua! mti*r block correlation, which exists in received blocky image. 
Second algorithm fiirtliei r<Hluc«is the block- boundary discontinuities in blocky image 
by capturing and reim>viiig the discontinuities in the DCT domain. The performance 
of both the algeirithms improves as compression ratio increases. These DCT domain 
algorithms an* romputatimsally efficient since fast algorithms for the computation of 
DCT are available. In third algorithm for block-boundary discontinuity reduction, the 
discontinuities are raptured in the same way but it uses the SVD based filtering scheme 
for their reduction. It r<Hhires the blocking artifacts considerably but its computational 
complexity is slightly more biH’iiuse HVI) algorithm has to be applied to each of the 
monotone b}f»r.k. Thus, this algorithm is very effective for reducing the blocking artifacts 
but its performance in t(«rms «»f I’HNH and computational complexity is slightly poor 
as compared to first ajjproarh of disr<intiimity reduction. Fourth algorithm reduces the 
blocking artifacts by eliminating some DCT coefficients of concatenated block of two 
adjacent blocks. This DCT domain algorithm has low computational complexity, as no 
filtering is reejuired. 'The availability of fast algorithm for DCT computation makes this 
algorithm suitable for h»w bit rate coding applications. The selection of first row and first 
column of the DC'F of concateiiaf block further reduces the computation complexity. 
This comput ationally tdficient algorithm also gives very good result in terms of PSNR, 
discontinuity measures and visual {|iiality. 

The performance analysis and simulation results show that the proposed techniques 
sie very competitive as comjjared to previously published works. 

6-4 Future Scope 

Tor the hybrid scheme, some more computationally efficient algorithm as suggested in 
Laptev 5 can be used in monotone area. In signal adaptive filtering for edge area, the 
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trade off between quality and complexity is required. 

For ID notch filter design, several well-known techniques are available in the litera- 
ture but there is a very little work in case of 2D notch filer. The optimum method for 
the design of 2D multiple-notch filter is required. The uses of this filter in some other 
applications have to be investigated. 

As we discussed in this thesis, the transform domain algorithm are very effective for 
postprocessing. More investigations are required to develop some more computationally 
efficient algorithms. The processing of DOT components due to edges may give smooth 
edges. This can be investigated for the further development of these algorithms. 

Finally, the testing of these algorithms for the postprocessing of still image and low 
bit rate video sequence in real time is required. 

Alternative DCT based coder such as a coder based on embedded zero tree scheme or 
a coder with variable size and variable shape blocks, can remove some of the drawbacks 
of conventional DCT based coder. 



Appendix A 


Standard JPEG Encoding 


A.l Introduction 

There is an increasing eflfort to achieve standardization in image compression. Members 
from both the International Telecommunication Union (ITU) and the International 
Organization for Standardization (ISO) are working together to establish a joint inter- 
national standard. As a result, IS (International Standard) 10918 has been established 
for both international organizations. This standard is known as JPEG, the Joint Pho- 
tographic Experts Group. The goal of JPEG has been to develop a general method 
for image compression, which will be as close as possible to the state of the art in 
image compression. JPEG includes two basic compression methods: a DCT-based lossy 
compression method and a predictive method for lossless compression. In this Appendix, 
an overview of the lossy JPEG method is presented. 

JPEG has been recognized as the most popular and efficient coding scheme for 
continuous tone still images, both monochrome and color. JPEG specifies one or more 
algorithm for image compression for each of the various modes. The four encoding modes 
are: 

• Sequential encoding: Sequential mode is the most common mode of operation. The 
image is encoded in raster scan fashion left-to-right/top-to-bottom and is based 
on DOT. The baseline system belongs to this category. 
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• Progressive encoding: The image is encoded in multiple scans at the same spatial 
resolution for applications in which the channel bandwidth is narrow and, therefore, 
the transmission time may be long, but the viewer prefers to view the image 
contents in multiple coarse-to-fine stages. 

• Lossless encoding: The image is encoded to guarantee exact recovery of every 
source sample value. The lossless method within JPEG is fully independent of 
transform-based coding. Instead, it uses differential coding to form prediction 
residuals that are then coded with either a Huffman coder or an arithmetic coder. 

• Hierarchical encoding: The image is encoded at multiple spatial resolutions, so 
that lower-resolution images may be accessed and displayed without having to 
decompress the image at a higher spatial resolution. 

Required coding techniques for JPEG are based on 2D (8 x 8) DOT. The DCT 
was chosen for two reasons, compression capability and the availability of commercial 
hardware chips. JPEG works best on natural images (scenes). It compresses the red- 
green-blue parts of a color image as three separate grayscale images - each compressed to 
a different extent, if desired. This is a widely used method of image compression which 
is based on performing the Discrete Cosine Transform (DCT) on 8 x 8 blocks of pixels. 
The JPEG standard prescribes a strict format for the encoded data stream, thereby 
allowing the use of a simple decoder to reconstruct the image. The JPEG method is 
designed to exploit limitations of the human eye. The information that is not perceived 
well by the eye is discarded. 

It is based on the fact that human eyes are more sensitive to small changes in 
brightness than in color. JPEG compression relies on several things. 

• Mostly the important signal energy of a picture lies at low frequencies. 

• The human eye has less sensitivity to representation errors in high frequencies. 

• In a small 8x8 block only a few important frequencies are present. 
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JPEG uses lossy compression scheme in which the data is thrown away during 
compression. The discarded data is in the form of higher frequency spatial information 
in the image, which is less visible. Most pictures usually have large areas of only low 
frequency components (flat borders, out-of-focus areas, sky, etc). JPEG also relies on 
limitations of the human eye’s ability to perceive detail by not throwing away high 
frequencies, but just coding it crudely and letting your eye “fill-in” the proper details. 
It has also the ability to only encode the “important” frequencies although not many 
encoders take advantage of this. When this compressed data is stored as a file and 
then re-opened, the image exhibits error, or JPEG artifacts. K the compression ratio is 
large, these artifacts show up as clearly visible blocking, with patterns sometimes seen 
in the blocks. The baseline JPEG algorithm partitions the image into 8x8 pixel blocks. 
The pixel data in each block are processed using a discrete cosine transform (DCT), 
which generates a frequency representation of the 8x8 block. The DCT coefficients are 
quantized, and the integer coefficients Huffman coded. The quantization eliminates DCT 
coefficients with small amplitudes, thus discarding the higher frequency data that these 
coefficients represent. This loss of data by quantization and coding is responsible for 
the high compression ratios that the JPEG scheme is capable of, but is also responsible 
for varying degrees of loss of image information. The block diagram of the baseline 
JPEG compression scheme is shown in Fig. A.l where as its simplified version is given 
in Chapter 1. The baseline algorithm consists of the following six stages: 

1. Transformation in color space. 

2. Subsampling. 

3. Discrete Transform Coding. 

4. Quantization. 

5. Data compression. 

6. File construction. 
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Figure A.l: Block diagram of the baseline JPEG compression scheme. 
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A. 2 Transformation in color space 

JPEG sets no restriction on the type of the input color space. Instead, it views each 
image as a collection of image components. The image is first transformed into a more 
suitable color space. For color images, the RGB data is transformed into luminance / 
chrominance color space. A widely accepted color scheme is the YCbCr. The compo- 
nents are defined by: 

Y = 0.2997? -H 0.587C? + 0.114B 

Cb = 0.1687i? - 0.3313G + 0.5B 

Cr = 0.5R - 0.4187G - 0.0813B 

Where, R, G and B represent the original Red, Green and Blue components respec- 
tively. The luminance component (Y) is the grayscale picture, which can be used directly 
as a black and white version of the image. The chrominance axes (Cb and Cr) contain 
all of the color information. Using our prior knowledge that the human visual system 
is more sensitive to changes in brightness, we can predict that an acceptable image can 
be reconstructed using fewer Cb and Cr samples than Y samples. Therefore, the three 
axes are processed separately, with the luminance axis at a much higher quality than 
the two-chrominance axes. 

It is not essential to transform the color space to luminance / chrominance. The 
JPEG algorithm would work on the original RGB values, however the compression will 
be reduced, since all three components would have to be coded at the higher luminance 
quality. 

A. 3 Subsampling 

Image with multiple components can be stored and processed either with or without 
data interleaving. This stage is also optional, and involves averaging together groups 
of pixels. Usually, the two color components are ‘downsampled’ at ratios of 2:1 that is 
every two adjacent pixels are averaged into one pixel value. The luminance component is 
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left at full resolution. By simple calculations, each component (Y, Cb and Cr) contains 
1 /3rd of the data. If Cb and Cr are downsampled at 2:1, they are reduced by half to 
l/6th of the original data. Thus the corresponding data volume becomes: 1/3 + 1/6 
+ 1/6 = 2/3 and hence is immediately reduced by l/3rd with almost no decline in 
perceived picture quality. 

These two stages mentioned above are optional and are used only as a form of 
preprocessing of the image and setting it up for the actual compression using JPEG. 


A. 4 Discrete Cosine Transform Coding 


DCT is a lossless, reversible process, which can decorrelate the image signal and achieve 
energy compaction. It can therefore reduce the redundancy in the image signal before 
transmitting through a band-limited channel, or storing in a memory-limited media. 
Thus key to the compression process is the Discrete Cosine Transform (DCT). The 
basic operation performed by this transforms is to take a signal and transform it from 
one type of representation to another. 

The pixel values of each component from the previous step are grouped into blocks, 
and the Discrete Cosine Transform (DCT) is used to transform the block into frequency 
domain coefficients. Generally, an 8 x 8 block of pixels is used, but this is purely 
arbitrary - it is mainly dependant upon the power of the software / hardware used to 
apply the DCT. The data points in each block are numbered from (0,0) in the upper 
left comer to (7,7) in the lower right comer. The DCT produces a new 8x8 block of 
spatial frequencies, F(u,v) using the formula; 


r./ N 2 , , / TT{2m+l)u)^ 7r(2n-f l)u)^^ 

F(w,t;) = -c(u)c(w)[Xl 5Z/(m,n)*co5( — )cos{ — )] 

^ m=0 n=0 


(A.l) 


where c{w) = for w = 0] c{w) = 1 for u; = 1, 2, ...7. 

After the transformation, the set of coefficients represent successively higher-frequency 
changes "within the block in both the x and y directions. F(0,0) represents the rate of no 
change in either direction and is known as the DC coefficient. It is also the average of the 
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8x8 input values. This allows us to separate the much more noticeable low- frequency 
information from the higher frequencies - which contain the ‘fine detail’ and can be 
removed without too much picture degradation. After DCT, as the rows and columns 
move away from the origin, the coefficients in the transformed DCT matrix begin to 
represent higher frequencies, with the highest frequencies found in the lower right comer 
of the matrix. Recalling the fact that most images have their important signal energy at 
low frequencies and that the human eye has less sensitivity to representation errors in 
high frequencies we can now identify which part of the image can be discarded without 
much change in the quality of the image. 

DCT is a lossless transformation that doesn’t actually perform compression. It sets 
up the base for the “lossy” stage of the compression. 

A. 5 Quantization 

The actual image compression is performed by quantization. Quantization is simply 
the process of reducing the number of bits needed to store an integer value by reducing 
the precision of the integer. The encoded image must include the 64 entries in the 
quantization- weighting matrix, which specify the stepsizes that were used to quantize the 
different DCT frequency components. A su^ested quantization matrix was developed 
by subjective visual trials and is included in Annex K of the JPEG specification although 
it is not stored as a default in the decoder (it must be transmitted). The table is stored 
in the JPEG file header so that the encoder can use adjusted coefficients for particular 
images. Most commonly, there is one table for the Y data, and another one for the Cb 
and Cr components. Many encoders use this suggested QM and scale it up or down based 
on a user-supplied “quality factor” to increase or decrease the amount of compression. 

Since JPEG is a lossy compression scheme, the reconstructed image differs from 
the original image, which results in some visible distortion. The key to successful 
JPEG compression, given a fixed encoded bit rate or file size, is the minimization of 
the amount of distortion that can be perceived by the human eye at that bit rate. 
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To compress a picture using JPEG, we must jBrst divide it into 8x8 blocks, then 
perform a two-dimensional DCT on each block. The result is a set of 8 x 8 blocks 
of DCT coefficients, Cijk, where i and j range from 0 to 7, and k ranges over the 
number of blocks. The coo position in each block corresponds to the low frequencies 
(DC) of the original image block in both horizontal and vertical directions while the Cjj 
coefficient contains the high frequency content of that block in the original image. The 
JPEG uses uniform midtread quantization to quantize the various coefficients. For lossy 
compression, each coefficient is divided by the corresponding quantization matrix entry 
and then round to the nearest integer: Uijk = round{cijk/qij). (In JPEG, this quantized 
data is then losslessly compressed using Huffman or arithmetic coding.) The decoder 
reverses this process by multiplying the quantized coefficients by the quantization matrix 
and performing the inverse DCT. 

Assuming the standard 8-bit quantization matrix, the least distortion and lowest 
compression are obtained with a matrix of all ones while the highest distortion and 
highest compression occur when using a QM filled with the maximum value 255. Each 
frequency component is divided by a separate quantization coefficient and rounded to the 
nearest integer. Each coefficient determines the extent to which a coefficient is weighted. 
A higher weighting means an increased loss of information. These values are the most 
varied component of the JPEG algorithm and the best results axe hard to determine. 
CCITT standard recommendation lists the example of a luminance quantization table 
given in Table A.l. 

This table is based upon “psychovisual thresholding” , and has apparently been used 
with good results on 8-bit per sample luminance and chrominance images. Since many 
experiments in perception have shown that the human visual system has different sensi- 
tivities at different frequency bands, it is not surprising that subjective trials would result 
in this table. Prom this table it can be observed that as we move from the DC coefficient 
to the higher-order coefficients the step size increases. Because the quantization error 
is an increasing function of the step size, more quantization error will be introduced 
in the higher-frequency coefficients than in the lower-frequency coefficients. The lower 
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Table A.l: Standard quantization matrix for JPEG 


16 

12 

10 

16 

24 

40 

51 

61 

11 

12 

14 

19 

26 

58 

60 

55 

14 

13 

16 

24 

40 

57 

69 

56 

14 

17 

22 

29 

51 

87 

80 

62 

18 

22 

37 

56 

68 

109 

103 

77 

24 

35 

55 

64 

81 

104 

113 

92 

49 

64 

78 

87 

103 

121 

120 

103 

72 

92 

95 

98 

112 

100 

101 

99 


Table A.2: Quantization matrix at quality factor of 10 


80 

60 

50 

80 

120 

200 

255 

255 

55 

60 

70 

95 

130 

255 

255 

255 

70 

65 

80 

120 

200 

255 

255 

255 

70 

85 

no 

145 

255 

255 

255 

255 

90 

110 

185 

255 

255 

255 

255 

255 

120 

175 

255 

255 

255 

255 

255 

255 

245 

255 

255 

255 

255 

255 

255 

255 

255 

255 

255 

255 

255 

255 

255 

255 


weights at top and left also indicate our generally higher sensitivity to the low ihequency 
components of images. In practical terms, this means that we should choose different 
weights for different frequency bands based on how well we are able to perceive distortion 
in that band (how our evaluation of image quality will be affected by loss of detail in 
that band). 

Most existing coders use simple multiples of this example, but the values are not 
claimed to be optimal. The common JPEG encoding practice is the use of a “quality 
factor” from 1 to 100. For a quality factor of 50, the suggested QM is used; for qf > 
50 (higher rate, lower distortion), the suggested QM is multiplied by (100-qf)/50; for 
qf < 50 (lower rate, higher distortion), the suggested QM is multiplied by 50/qf. The 
scaled matrices are rounded and clipped to the range [1, 255] before being used to encode 
images (because the QM is transmitted and used for decoding, while the quality factor 
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Table A. 3: Quantization matrix at quality factor of 90 


3 

2 

2 

3 

5 

8 

10 

12 

2 

2 

3 

4 

5 

12 

12 

11 

3 

3 

3 

5 

8 

11 

14 

11 

3 

3 

4 

6 

10 

17 

16 

12 

4 

4 

7 

11 

14 

22 

21 

15 

5 

7 

11 

13 

16 

21 

23 

18 

10 

13 

16 

17 

21 

24 

24 

21 

14 

18 

19 

20 

22 

20 

20 

20 


is not). For example, the matrices given in Table A.2 and Table A.3 are obtained for 
quality factors of 10 and 90, respectively; 

The bit-rate (entropy encoded size/number of pixels) is a function of the exact image 
being encoded. If we have a target rate in mind, we must iteratively try different quality 
factors, although the process is made easier because the rate is a monotonic function of 
the quality factor. However, this method leaves much to be desired, since it does not 
calculate or optimize the amount of visible distortion at the desired rate. 

A.6 DC Coding and ZigZag Sequence 

After quantization, the DC coefficient is treated separately from the 63 AC coefficients. 
The DC coefficient is a measure of the average value of the 64 image samples. Because 
there is usually strong correlation between the DC coefficients of adjacent 8x8 blocks, the 
quantized DC coefficient is encoded as the difference from the DC term of the previous 
block in the encoding order. This special treatment is worthwhile as DC coefficients 
frequently contain a significant fraction of the total image energy. Re-ordering of 
non-zero coefficients is performed to better exploit the statistical occurrence of zero 
valued coefficients in an energy compacting transform. The most popular coefficient 
re-ordering is that of incremental zigzag scanning of DCT coefficients from its DC value 
up to its highest frequency components. This process will reduce the number of code 
words needed to code non-zero coefficients. Thus coefficients of the AC components 
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axe arranged in a “zigzag sequence” and then are encoded using two different methods. 
The first is run-length encoding of zero values. This method takes advantage of the 
large number of zeroes that exist in the bottom half of the matrix. The second is what 
JPEG calls “Entropy Coding”. This involves sending out the coeflacient codes using 
either Huffman codes or arithmetic coding depending on the choice of the implementer. 
However, baseline JPEG implementation uses Huffman coding only. 

A. 7 Entropy Coding 

Entropy encoding is the last step of the sequential JPEG compression algorithm. An 
entropy encoder further compresses the quantized values losslessly to give better overall 
compression. It uses a model to accurately determine the probabilities for each quantized 
value and produces an appropriate code based on these probabilities so that the resultant 
output code stream will be smaller than the input stream. Two approaches are specified 
by the JPEG standard. The baseline system uses simple Huffman coding while the 
extended system uses arithmetic coding and is suitable for a wider range of applications. 

The number of previous zeros and the bits needed for the current numbers amplitude 
form a pair. Each pair has its own code word, assigned through a variable length code (for 
example Huffman, Shannon-Fano or Arithmetic coding). In baseline JPEG, Huffman 
coding is also used for AC coefficients. JPEG outputs the code word of the pair, and 
then the code word for the coefficient’s amplitude (also from a variable length code). 
After each block, JPEG writes a unique end-of-block sequence to the output stream, 
and moves to the next block. When finished with all blocks, JPEG writes the end-of-file 
marker. 


A. 8 File Construction 

Finally, a header is added at the top of the file, containing all of the compression 
parameters needed to reverse the encoding process. 
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Figure A.2; Format of JPEG bit stream. 


A. 8.1 Overview of the JPEG bit strecim 

A “Frame” is a picture, a “scan” is a pass through the pixels (e.g., the red component), 
a “segment” is a group of blocks, a “block” is an 8x8 group of pixels. The format for the 
JPEG bit stream is shown in Fig. A.2. The frame header and the scan header contain 
the following information: 

Frame header: 

• sample precision 

• (width, height) of image 

• number of components 

• unique ID (for each component) 

• horizontal/vertical sampling factors (for each component) 

• quantization table to use (for eadi component) 
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Scan header: 

• Number of components in scan 

• component ID (for each component) 

• Huffman table for each component (for each component) 

The following miscellaneous information can occur between headers: Miscellaneous 

• Quantization tables 

• Huffman Tables 

• Arithmetic Coding Tables 

• Comments 

• Application Data 

JPEGs are suitable for the compression of true color or grayscale images of pho- 
tographs and realistic scenes. But the disadvantage with JPEG is that it is not suited 
for artwork with sharply defined lines, lettering and simple cartoons. This is because 
JPEG tends to blur the borders between one color and another. 
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